Output-driven SIEM — 13 years later

Output-driven SIEM! Apart from EDR and SOC visibility triad, this is probably my most known “invention” even though I was very clear that I stole this from the Vigilant crew back in 2011.

Anyhow, I asked this question on X the other day:

So, what year is this? Let me see … 2025! Anyhow, get a time machine, we are flying to 2012…. whooosh….

… we landed … no dinosaurs in sight so we didn’t screw the time settings.

Now, WTH is “output-driven SIEM”? Back then, I said that it stands for “deploying your SIEM in such a way that NOTHING comes into your SIEM unless and until you know how it would be utilized and/or presented.” This is not about “don’t collect unless you detect” as some have mis-interpreted it, because you may have some logs for context, or for IR or even … gasp … for compliance. Just not for the sheer heck of it 🙂

What changed?

Let’s see what is different 13 years later, in 2025.

1. “Output driven SIEM” is still largely a good idea in 2025. Don’t collect unless you have “the WHY for it” sounds like common sense, that most uncommon of all senses. It is also closely related to thinking about the use cases around SIEM, something I also spent years evangelizing.
2. Weirdly, I intuit that there was a time between 2012 and today when “just collect it” was almost workable (storage got cheap before log volumes got up). However, I think in 2025, today’s logs overrun today’s storage just as reliably as 2010 logs overran 2010 storage.
3. SOAR (when it came around 2015) was a way for people who refused to practice “output driven SIEM” (and/or tune the detections) to survive for a bit longer, without changing the fundamentals. SOAR saved some people with “input-centric SIEM” for a bit, just as it saved some people with shit detections (also for a bit)
4. What about “AI SOC”? Similar to SOAR … but also different! I think “AI SOC” will help people with poor detection decisions more than it will help people with poor [read: too wide open] collection decisions. Ultimately, if you collect, you pay (decentralized trick may work for some people and for some logs some of the time).
5. Also, “Output driven SIEM” is still a part of a healthy answer to alert fatigue in 2025. If you collect lots of random stuff and run some random detections on it, you will have alert fatigue. And you won’t have a good time. So be “output-driven” … think what you want to see at the … you got it… OUTPUT! AI does not change this, frankly.

What to do?

What was the best way to architect “output-driven SIEM” yet still not lose the data you may need later for some purpose? Before and around 2010, it was almost always a two-tier system: a SIEM + central log management (CLM). Here are some of my examples from 2009. Is 2 tier system SIEM+CLM still the best answer?

Naturally, we have way more log storage options in 2025 (Clickhouse anybody), but there is a fundamental truth that remains: if you collect a log and save it, somebody somewhere pays for a hard drive. Nothing has ever charged it, and probably never will.

Look, I like modern one-tier stuff as much as the next person. Google SecOps (formerly Chronicle) can store logs hot for a year (or more!) in a very affordable fashion. But very obviously “one tier for all logs” is not an answer for all SIEM and logging questions (at the very least, because compliance retention requirements drives the quest for ever-cheaper storage, eventually devolving to “write-only log retention”).

So, to conclude, my “3AM answer” for how to architect an output-driven SIEM in 2025 is: look at modern one-tier approach (like, well, our SecOps), but if that does not fit, go back to the classics (cheap/broad log management tool or a data lake + a SIEM based on “output-driven” model).

What to expect?

My analysis is usually grounded in reality, some say too grounded (read: not ambitious enough). So let me try to act out one of my impulses here. I think AI agents in SOC have a chance — in the future — to shift us from the “output-driven SIEM” to “outcome-driven SOC.” What I — hypothetically — mean by this is that a human operator defines a strategic security outcome for a SOC, and then agents get to work and decide the collection, detection and response datasets, tooling and practices. And then do the work! Amazing, eh? Well, there is a caveat…

And it is … can we have this IRL? Obviously not today (we are far from it), but ask me again in … 3 years. My time machine does not have a forward button, so I have to “crystal ball it”… otherwise, just “organically” wait for 2028. Humans to decide what to do … and then I want the robots to do it!

Eventually.

Related blogs:

• Debating SIEM in 2023, Part 1
• Debating SIEM in 2023, Part 2
• 15+ Years of Loading Threat Intel into SIEM: Why Does This Still Suck?
• Decoupled SIEM: Brilliant or Stupid?
• Log Centralization: The End Is Nigh?
• SIEM Content, False Positives and Engineering (Or Not) Security

Output-driven SIEM — 13 years later was originally published in Anton on Security on Medium, where people are continuing the conversation by highlighting and responding to this story.

*** This is a Security Bloggers Network syndicated blog from Stories by Anton Chuvakin on Medium authored by Anton Chuvakin. Read the original post at: https://medium.com/anton-on-security/output-driven-siem-13-years-later-c549370abf11?source=rss-11065c9e943e------2