The Security Operations Stack

A SIEM (security information and event management) is a platform that collects log and event data from across an environment, normalizes it into a common format, applies correlation rules to look for patterns that indicate an attack, and retains the data so investigators can search it later.

A SIEM does four jobs, and separating them matters, because most disappointment with SIEM traces back to buying it for one of them and being judged on another:

• Ingest and normalization: getting data in from many sources and making a Windows logon and an Okta logon comparable.
• Correlation and detection: applying rules across sources so that a failed login in one system and a privilege change in another can be recognized as one story.
• Retention and search: keeping the data long enough that an investigation into something that started four months ago is possible at all.
• Reporting: demonstrating to an auditor that monitoring exists and is operating.

Reporting is a large part of why many SIEMs are purchased, and it is a legitimate reason. It is also why a SIEM can be judged a success by compliance and a failure by the SOC in the same quarter, since a deployment tuned for evidence of coverage is not the same as one tuned for detection.

The products most people mean by SIEM are Splunk, Microsoft Sentinel, IBM QRadar, Elastic, and Google Security Operations, alongside the analytics-led entrants Exabeam and Securonix.

EDR (endpoint detection and response) is software that runs on endpoints such as laptops, servers, and workstations. It records what happens on the host, including process execution, file changes, registry activity, and network connections, applies detection logic locally and in the cloud, and gives responders the ability to act on the host, such as isolating it from the network or killing a process.

XDR (extended detection and response) extends that model beyond the endpoint. The intent is to correlate detections across endpoint, identity, email, network, and cloud within one platform, so that a single story spread across four telemetry sources is presented as one incident rather than four alerts.

The complication with XDR is that the definition isn't settled. Some XDR products are genuinely multi-domain correlation platforms. Others are an EDR with additional integrations, marketed under the newer label. The distinction is not visible from the category name, and the practical test is to ask which telemetry sources the correlation actually spans, and whether the correlation is performed by the product or by a rule you have to write yourself.

In product terms the EDR category is CrowdStrike, SentinelOne, and Microsoft Defender for Endpoint; the XDR label attaches to Microsoft Defender XDR, Palo Alto Cortex XDR, and Cortex XSIAM, which is where the definitional question above is worth asking directly.

They sit at different layers, and the overlap between them is real but partial. SOAR (security orchestration, automation, and response) is the fourth term in this group and it belongs to a different category than the other three: SIEM, EDR, and XDR find things, and SOAR acts on what they find by running predefined workflows across your tools. A fuller treatment of the category sits in this guide to SOAR.

The overlap causes genuine confusion in buying conversations, and the useful framing is that SIEM is horizontal and shallow, EDR is vertical and deep, XDR tries to be both across a fixed set of domains, and SOAR is the automation layer sitting on top of whichever of them you have. On the SOAR side that usually means Splunk SOAR, with ServiceNow carrying the case and ticketing half of the same workflow.

Playbooks decay because they encode assumptions about systems that keep changing, and nothing tells you when an assumption stops being true. The mechanism is well understood and rarely written down.

The main decay mechanisms:

• Integrations drift: an API version changes, a field is renamed, an authentication token rotates, and a step that worked yesterday silently returns nothing today. The playbook often keeps running and produces a confident wrong result rather than an error.
• The environment moves underneath the logic: A playbook that isolates a host assumes the host is managed by the agent it knows about. Estates change, new tooling arrives, and coverage assumptions quietly stop holding.
• Detections get tuned, and playbooks do not: when a rule is narrowed or widened, the playbook that consumes its output is rarely revisited, so the automation now runs on a differently shaped population of alerts.
• The branch coverage was never complete: playbooks are written against the cases the author had seen. The long tail of unusual cases either falls out of the flow or gets forced down a branch that does not fit.
• The author leaves: playbooks are software, and they are usually maintained like documentation. When the person who wrote them moves on, the willingness of anyone else to modify a working-looking automation drops sharply.

The compounding effect is what makes this expensive. Each of those mechanisms is individually small and individually fixable, and together they mean a playbook estate needs continuous maintenance effort that is almost never budgeted at purchase. Teams commonly report needing dedicated engineering capacity just to keep an existing playbook set alive, which is a cost that doesn't appear in any category comparison.

Two of the limits are structural rather than maintenance problems, and no amount of upkeep resolves them. A playbook can only handle a situation somebody anticipated, and a playbook cannot decide anything the author did not already decide, and a playbook can only handle a situation somebody anticipated, and it can't decide anything the author didn't already decide. Approaches that try to work around that constraint are covered in the AI SOC section.

EDR catches most of what touches an endpoint, which covers the bulk of execution and persistence activity and very little identity activity, that is a real advantage and a bounded one: endpoint telemetry is the richest single source available, and an attacker who signs in to a cloud service with a stolen credential never touches an endpoint agent at all.

What EDR can't do falls into four categories, and each one is a real reason SIEMs continue to exist:

• Anything that never touches a managed endpoint: identity attacks against cloud services, business email compromise, abuse of a SaaS admin console, and activity on unmanaged or unmanageable devices are invisible to endpoint agents.
• Correlation across sources: A failed VPN login, a password reset, and a mailbox rule creation are individually unremarkable and collectively a story. Only something that sees all three can tell it.
• Retention and retrospective search: EDR telemetry retention windows are commonly short, and investigations frequently need to look further back than the window allows. When a campaign started months ago, the question "was this happening in March" needs data that is still there in March.
• Evidence for people who are not the SOC: auditors, regulators, and legal teams need retained, searchable records across systems, and that requirement is largely independent of detection quality.

Many organizations buy a SIEM for detection, get most of their detections from EDR anyway, and end up using the SIEM primarily for correlation, retention, and compliance. That's not a failure, but it is a different value proposition than the one usually sold, and teams that recognize it early tend to make better decisions about how much they spend on ingest.

Log management is the collection, storage, indexing, and retrieval of log data. A SIEM is log management plus a detection and correlation layer on top. Every SIEM contains a log management system, and not every log management system is a SIEM.

The difference in practice shows up in three places:

• Intent of the data model: log management stores what arrives. A SIEM normalizes it so that events from different sources can be compared and correlated, which is expensive and is most of thedeployment work.
• Whether anything is watching: log management is a store you query. A SIEM continuously evaluates incoming data against rules and produces alerts.
• Cost shape: a SIEM charges for indexing and correlation as well as storage, so a log store is generally the cheaper home for high-volume, low-detection-value sources. This is why many teams route those sources to a log store and forward only the security-relevant subset to the SIEM.

Tiered logging, sometimes called a security data lake, has become common because ingest volume grows faster than detection value, and the trade-off is real and worth stating: data that does not reach the SIEM cannot be correlated in real time, so the routing decision is a detection decision, not just a cost decision. CrowdStrike LogScale, Datadog, and Wazuh are the stores this pattern most often routes to.

Build in the order that the detection-to-response path runs, because each layer is worth much less without the one before it, and buying out of sequence is the most reliable way to end up with tools that nobody uses.

• Visibility first: endpoint telemetry and identity logs, because together they cover the two things nearly every attack touches. Without these, everything downstream is guessing.
• Then response capability on what you can already see: the ability to isolate a host and disable an account matters more than an additional detection source, since a detection you cannot act on is only a notification.
• Then aggregation and retention: A place to keep the data and search across it, which is what turns single alerts into investigations and satisfies the audit requirement at the same time.
• Then correlation and detection engineering: rules that span sources, and somebody whose job is to write and tune them. This is the point at which the queue starts getting better instead of bigger.
• Then automation: only once the process is stable enough to be worth encoding. Automating an unstable process makes the instability faster.
• Then breadth: network, cloud posture, email, and the rest, added in the order your actual incidents suggest rather than the order the market suggests. External context, from a feed such as Recorded Future, belongs here rather than at the start, since intelligence about what attackers are doing elsewhere is only actionable once you can see what is happening in your own estate.

The mistake this ordering is designed to prevent is the common one of buying a correlation and automation layer before there is either data worth correlating or a process worth automating, and a team in that position generally spends its first year building integrations rather than finding attacks.

One more sequencing note. Adding a telemetry source adds alerts, and alerts consume people. Every expansion of visibility should be paired with a decision about who works the resulting queue, or the practical effect of better coverage is a longer backlog.