Loading...
Loading...
Detection as code is the practice of writing detection logic as version-controlled code — Sigma or YAML rules in Git, peer-reviewed through pull requests and shipped by a CI/CD pipeline. The 2025 State of SIEM Detection Risk report found the average SIEM ingests enough telemetry to cover 90% of MITRE ATT&CK and carries detections for 22% of it.
Your pipeline is green. Your MITRE ATT&CK coverage is 22%. Both of those are true at the same time, and only one of them is a delivery problem.
Detection as code takes detection logic out of the SIEM console and treats it like software. A rule gets written in a text format such as Sigma, YAML, or Python, committed to Git, reviewed by a peer through a pull request, checked by unit tests in CI, and deployed by a pipeline.
NVISO's detection lifecycle names six phases for the same loop: requirements, design, development, testing, deployment, and monitoring.
What it replaces is worth remembering. A rule used to live in a UI, edited by whoever had the tab open, with no diff and no way to answer why a threshold changed in March.
Now every rule has an author, a commit, a test, and a rollback.
Detection engineering platform tooling has settled into a handful of approaches:
| Approach | What it standardizes | Best for |
|---|---|---|
| Sigma | Vendor-neutral YAML for log-based detection; converts to KQL, SPL, EQL | Portability across SIEMs |
| YARA / YARA-X | File and memory pattern matching | Malware and content-based detection |
| Python rules (Panther, Datadog) | Detection logic as code rather than a schema | Nested data a YAML schema can't reach |
| Native repo conventions (Elastic) | Published directory structure and contribution guide | Single-platform shops |
| Terraform, GitHub Actions, GitLab CI, Jenkins | Lint, test, and deploy rules as infrastructure | Teams already running GitOps |
A Sigma rule is a vendor-neutral detection written in YAML that describes a log pattern once and converts to any SIEM's query language. The SigmaHQ repository publishes several thousand community rules, and sigma-cli or pySigma convert them into Splunk SPL, Microsoft Sentinel KQL, or Elastic EQL. SOC Prime's Uncoder AI does the same translation in a browser for teams not ready to run their own converter.
Sigma is the portability layer. Something still has to run the translated query.
Detection as code is to detection rules what infrastructure as code is to servers: the artifact moves out of a console into a versioned text file, and a pipeline becomes the only route to production. The difference is what happens when something breaks. A Terraform plan fails loudly when the resource is wrong. A detection rule whose source field stopped arriving passes every check in the pipeline and simply never fires.
That is good engineering, and none of it is what this post is about.
Detection coverage is stuck because detection as code governs delivery, and the shortage sits upstream of delivery. A pipeline moves rules faster and more safely. It has no opinion about which rules should exist, and a CI/CD pipeline with nothing in its queue is a very well-tested empty repository.
CardinalOps' fifth annual State of SIEM Detection Risk report measured the shortfall across production environments running Splunk, Microsoft Sentinel, CrowdStrike LogScale, and Google SecOps. Its 2025 sample found average coverage at 22% of ATT&CK techniques, up from 19% the year before. The telemetry already flowing into those same SIEMs was enough for 90%.
One number in that report should worry you more than the headline. Against the ten techniques that appear most often in real intrusions, the average SIEM carried detections for four.
The report names three causes: adversary sophistication, sprawling IT complexity, and a detection engineering function it describes as "still reliant on error-prone manual processes and individual heroics." Only the third is addressable this quarter. The data arrived; the rules to read it were never written.
Threat intelligence, incident response, and threat hunting decide the detection engineering backlog. Detection engineers execute it. That dependency is the part most detection-as-code writing skips, because it leaves the function with no independent way to know which technique matters next.
SpecterOps' Joshua Prager and Emily Leidy made the argument most precisely in their 2022 work on prioritizing the detection engineering backlog. A threat hunter can form a hypothesis, research it, and write a query with very little external input.
A detection engineer cannot. The function needs external stimuli from other detection and response teams to prioritize accurately, or resources get spent in the wrong direction at the wrong time.
Most teams work the reverse of their ordering:
Read that against a typical backlog and the mismatch shows. Most queues run on Priority 2, because vendor threat reports arrive on a schedule and gap analyses don't. Teams write detections for the campaign in the headline while the technique that walked past their sensors last quarter stays unwritten.
Threat hunting is close to the only function in a SOC that starts itself. Detection engineering sits at the other end of that spectrum, which changes the automation question. Not whether a model can write a Sigma rule, but where the queue comes from, and whether your environment fills it or someone else's threat report does. That dependency is also what separates detection engineering from alert triage in daily practice.
AI can write detection as code rules for the first draft, and that is a smaller claim than it sounds. Language models handle the syntax half well. They translate logic between KQL, SPL, and Sigma, draft a query from a described behavior, and keep field names consistent across a rule set.
Tedious work, worth handing off. It was also never the bottleneck.
The hard half is knowing whether the logic fits your environment, and practitioners are direct about who holds that knowledge. On a 2026 r/threatintel thread, a detection engineer called commodity vendor rules a mixed bag: fine as a tuned baseline, worth replacing wherever they could do better. Their reason was that they know their environment better than their vendor does. The value, they said, sits in custom rules no vendor has ever written, the ones that catch a low-and-slow password guesser.
They're right, and the criterion they're using is the correct one. Any answer to this question that doesn't concede the point is selling something. Proximity to the environment is what makes a detection good, and no amount of model capability transfers that proximity from the person who has it.
There is a second bar underneath. On an earlier r/cybersecurity thread, a SOC analyst described their biggest frustration with inherited detections as being handed a signature with no reasoning attached: it is bad, trust us, we can't tell you why. Their conclusion was that detections without added context are a net negative to operations.
A machine can produce valid detection logic. The test is whether the rule arrives carrying the case that produced it.
You test a detection as code candidate the same way you should test a human-written one, with two replay checks most pipelines skip. Does it fire on the attack you actually missed, and how often does it fire during an ordinary week?
A green build answers neither. It tells you the rule parses, the fields resolve, and it matched a sample event someone committed alongside it. That is a syntax check wearing a lab coat.
Replay is the stronger instrument, and practitioners already run the manual version. One engineer in that detection-as-code thread described large-team practice as testing against existing alerts and use cases, throwing logs back at the rule with a replay tool. Two directions matter:
Both use data you already retain. Datadog's historical jobs run the same idea inside a shipping product, which is a reasonable sign the pattern is not exotic.
Adversary simulation — breach and attack simulation, or BAS — answers a different question and belongs alongside replay rather than in competition with it. Simulation exercises techniques you have not been hit by, which is the only way to learn anything about attacks you have never seen. Replay proves fidelity against the ones that already reached you. A team running both should not expect them to agree, and the disagreements are usually the interesting part.
Roughly 10% of deployed detection rules never fire because their data sources are misconfigured, their fields are missing, or their parsing is broken — and detection as code does not catch any of that. It confirms that a rule's syntax is valid. The field that rule reads could have stopped arriving three weeks ago and every check would still pass.
CardinalOps' 2025 sample measured that 10%. It is the best figure in five years of the study, down from 18%.
It still works out to roughly 16 dead rules in an average estate of 163. Simbian's 2026 estimate is that about 20% of detection rules stop firing within six months as telemetry drifts and tools get swapped.
The failure mode is quiet by construction. One field renamed during a SIEM upgrade takes every rule that reads it, with no error, no alert, and no failed build, because the syntax never changed.
You can find yours this afternoon. Query for every enabled rule with zero fires in 90 days, then check the field names in each against your current schema — the ones failing both are the dead set. Adopting a normalized schema such as OCSF or Elastic Common Schema shrinks the problem, since a field rename happens once at the pipeline rather than in every rule downstream.
The software analogy that gave detection as code its name stops being generous here. Real software engineering did not stop at continuous deployment. It added observability, regression detection, and deprecation. Detection content borrowed the delivery half of the discipline.
You measure MITRE ATT&CK coverage per technique, in four states, against one map. ATT&CK works as the coordinate system because every finding, alert, and gap resolves to the same technique ID, so offensive and defensive results land on one axis.
For any technique, three questions decide the state:
| State | Was it tested? | Did an alert fire? | Was the verdict right? |
|---|---|---|---|
| Covered | Yes | Yes | Yes |
| Triage Gap | Yes | Yes | No |
| Detection Gap | Yes | No | not reached |
| Untested | No | not reached | not reached |
Most coverage dashboards answer a narrower question than they appear to. "A rule exists for T1558.003" describes what is in the repository. Detection coverage requires that the technique was exercised, that something fired, and that whoever looked at it reached the right verdict.
A triage gap is a technique where the detection worked, the alert fired, and a human closed it wrong. It is the state nobody counts, and it is the expensive one — the intrusion carries on behind a green dashboard while the metrics show a cleared queue.
SANS' 2025 Detection and Response Survey found 73% of organizations naming false positives their leading detection challenge, up sharply from 64% the year before. That is the triage gap showing up where most teams can actually see it, as a staffing complaint.
It also means 22% is the optimistic number. Coverage counts techniques with a rule attached. Techniques where a rule fired and a human closed it wrong are not subtracted.
An AI detection engineering agent is software that occupies the authoring seat upstream of your pipeline. It reads what your security operations already produced, decides which detections are missing or noisy, and writes candidate rules for a human to approve. The pipeline you built still does the shipping.
Simbian's AI Detection Engineering Agent is one, and its inputs are the design decision worth examining. The primary engine is the AI SOC Agent and the AI Threat Hunt Agent. The SOC agent supplies false-positive and true-positive signal from real investigations; the hunt agent surfaces activity that never generated an alert at all. Offensive testing sits third, as the amplifier rather than the engine.
That ordering is the SpecterOps input principle running continuously rather than quarterly. Priority 1 material, generated by your own environment, instead of a queue filled by whichever threat report shipped this week.
A detection engineering agent fills the backlog from memory that outlives a single case, which is also why it is a separate agent rather than a SOC feature. An investigation and a hunt are each scoped to one session. Each can suggest a rule for what it just saw, then forget.
A detection engineering agent works across a corpus assembled from all of them. It tracks how many investigations a rule has driven, its false-positive and true-positive history, the entities involved, and whether a query it recommended earlier still holds. That cross-session history separates a suggestion from a judgment.
The output is a query you can deploy:
| SIEM | Query language | Agent output |
|---|---|---|
| Microsoft Sentinel | KQL | native |
| Splunk Enterprise Security | SPL | native |
| Securonix | SPOTTER | native |
| IBM QRadar | AQL | native |
| Any platform | Sigma YAML | always generated as the neutral baseline |
Review means reading rather than translating. Everything an agent learns lands in Context Lake™, the shared memory the agents read and write.
A human approval gate stops an agent from deploying a bad rule. Every candidate is replay-validated in both directions before anyone sees it: true-positive against the campaign telemetry that was missed, false-positive against a baseline window of the tenant's own traffic, using existing integrations rather than new simulation infrastructure.
On approval it deploys automatically, with rollback and an audit trail recording what drove the rule, when it shipped, and who signed it off.
Read that guarantee precisely, because it is easy to over-read in either direction. Never auto-deployed means never without a human. Deployment itself stays automatic. The agent is self-improving, not self-driving.
On evidence, there are no detection-engineering-specific results to cite. The mechanism is documented and the approval model is real. Read this section as a description of how it works, and wait for published numbers before counting it as evidence.
What has been measured is the quality of the signal feeding it. In NTT Data Japan's 2025 evaluation, Simbian's AI SOC Agent reached 94.9% agreement with human analysts on true-positive and false-positive judgment across 138 evaluated alerts. That result belongs to the SOC agent, and it matters here only because a backlog built from bad verdicts produces well-tested rules for problems you don't have.
Q: Is detection as code the same as detection engineering? No. Detection engineering is the discipline of deciding what to detect and building the logic to do it. Detection as code is one delivery methodology for that logic, covering version control, review, testing, and deployment. A team can practice threat detection engineering with no pipeline at all, and a team can run an excellent pipeline while writing very few detections.
Q: What tools do you need for detection as code? A Git host, a CI runner, and a rule format. In practice that means GitHub or GitLab for version control and review, GitHub Actions, GitLab CI, or Jenkins for testing and deployment, and Sigma YAML converted with sigma-cli or pySigma. SOC Prime's Uncoder AI covers translation in a browser for teams not ready to run a converter, and Terraform manages the rules as infrastructure if your estate already runs on it.
Q: Is detection as code worth it for a small team? Yes, and the threshold is lower than most teams assume. If your infrastructure already runs through Terraform and your deploys already run through CI/CD, a detection repository is one more pipeline rather than a new discipline, and a two-person team gets the same rollback, diff history, and review trail a twenty-person team gets. What does not scale down is the backlog: someone still has to decide what to write.
Q: Does detection as code replace my SIEM? No. The SIEM still ingests the data, runs the queries, and raises the alerts. Detection as code changes where the rule is authored and how it gets there, moving it from a console into a repository with review and rollback. Your platform's query language, its data model, and its licensing are all unaffected.
Q: Does detection as code help with compliance audits? Yes, and it is often why adoption gets funded. Change-management controls already require that every security-relevant change be documented and restorable. When rules live in Git, each commit carries an author, a timestamp, a justification, and a rollback point, so the audit trail is generated by the workflow rather than reconstructed for the auditor.
Q: Is it safe to let an AI detection engineering agent write detections? It is safe when the agent proposes and a person disposes. In Simbian's model every rule is a candidate that is replay-validated against the missed attack and against the tenant's own baseline before review, held for human approval, and deployed with rollback and a full audit trail. No mode exists in which a generated rule reaches a production SIEM without someone signing off.
Q: Do more detection rules mean better coverage? No, and the relationship can invert. CardinalOps found rule counts falling by nearly a quarter since 2022 while coverage rose, alongside a drop in broken rules, because fewer rules left more time for maintenance. Detection coverage is measured per technique against a map. Repository size tells you nothing about it.
The argument for detection as code is over, and it should be. Version control, peer review, and a tested deployment path are how detection content should have been managed all along.
It solved the half of the problem that was visible. The other half shows up as 22% coverage against telemetry that supports 90%, as four of the top ten attacker techniques going undetected, as roughly 16 dead rules in an average estate, and as a backlog ordered by whichever threat report landed most recently.
Every one of those is somebody's Tuesday.
So take a different question into your next planning cycle. Where do the next twenty detections come from, can anyone name the technique each one closes, and what would your team have to stop doing to write them? If you already know which twenty are missing and can't say who will write them, the constraint was never delivery — and it is worth booking a demo to see what an agent in the authoring seat produces.
