Loading...
Loading...
The short answer: Anthropic released Claude Opus 4.7 with the Cyber Verification Program and the first "production cybersecurity safeguards" in any Claude release. On Simbian's Cyber Defense Benchmark — 1,206 real attack-log investigations across 13 of 14 MITRE ATT&CK tactics — Opus 4.7 scored 30.9%. Opus 4.6 (five months older, no special cyber treatment) scored 44.5%. Sonnet 4.6 (cheaper) scored 36.9%. The cyber-ready Claude is the worst Claude for cybersecurity, and no frontier model passed the 50% threshold.
Anthropic announced Opus 4.7 on April 16, 2026 as the first Claude built for cyber work, with offensive-prompt guardrails and a Cyber Verification Program for legitimate researchers. We ran every current Claude through the Cyber Defense Benchmark (arXiv 2604.19533) — a defense-oriented cybersecurity LLM gym with deterministic ground truth, covering 13 of 14 MITRE ATT&CK tactics and 105 procedures chained into real kill-chains. Which Claude should a SOC actually run? The newest one finished last.
▶ Watch the webinar — Why LLMs Fail in the SOC A 30-minute walkthrough of the Cyber Defense Benchmark: how every frontier model was scored, per-tactic breakdowns, and why no LLM clears 50% coverage on real attack logs.
Fourteen frontier models, 1,206 investigations, 100K+ events per environment. Every model reasoned through SQL queries against real Windows attack telemetry, chained across 105 MITRE ATT&CK procedures. The passing threshold is 50% coverage. The leader is Opus 4.6 at 44.5%, short of the bar but the only model to pass 7 of 13 tactics. The top four slots all belong to Anthropic:
| Rank | Model | Coverage | Median cost | Tactics passing |
|---|---|---|---|---|
| 1 | Opus 4.6 | 44.5% | $2.71 | 7 of 13 |
| 2 | Sonnet 4.6 | 36.9% | $2.21 | 2 of 13 |
| 3 | Opus 4.8 | 32.7% | $1.69 | 1 of 13 |
| 4 | Opus 4.7 | 30.9% | $1.65 | 0 of 13 |
Every other frontier model (GPT 5.5, Gemini 3.5 Flash, GPT 5, DeepSeek 4 Pro, Kimi 2.6) sits below 27%. The category isn't saturated. It's failing.
Opus 4.7 is the model Anthropic explicitly built for cybersecurity. It ships with automated detection of high-risk prompts, the Cyber Verification Program, and product copy that names "cyber-grade safeguards." The marketing says cyber-ready.
The benchmark says otherwise. Opus 4.7 lands at 30.9% coverage and zero MITRE tactics passing, the worst Claude on the leaderboard. The likely mechanism: 4.7 optimizes for cheap, fast completion over thorough investigation. Its median investigation runs $1.65 at 243 seconds — the fastest and cheapest Claude, and also the least useful for SOC work. Opus 4.8 doesn't recover the gap. At 32.7%, it edges 4.7 but still trails the five-month-older 4.6 by 12 points.
Sonnet 4.6, Anthropic's mid-tier model at $2.21 per investigation, outscores both Opus 4.7 (30.9%) and Opus 4.8 (32.7%) by a clean margin. If you are paying flagship money for the best LLM for SOC work and reaching for the newest Opus, you are paying more for less coverage. Opus 4.6 leads coverage by 7.6 points over Sonnet for a 23% cost premium — defensible if coverage is the priority. Opus 4.7 trails Sonnet by 6 points at a 25% discount — not defensible at any seat-price. Every dollar of Opus 4.7 or 4.8 over Sonnet today buys less coverage, not more.
The skills that win on SWE-bench and coding benchmarks do not transfer to defense. SOC reasoning needs hypothesis generation under heavy noise, MITRE-coverage planning, and iterative SQL across 100K+ events per environment — none of which appear on offensive or general-purpose cybersecurity LLM evals. Defensive work also penalizes speed: Gemini 3 Flash, fastest model at 68 seconds, scores 14.6%. Investigation depth tracks investigation time. Anthropic's safeguards work as designed, but offensive-prompt guardrails do not add defensive skill. This is the load-bearing finding: the harness, the context, and the skills around the model matter more than the model. Simbian's record on the same benchmark, same scenarios, same data, wrapped in our harness, is 95% coverage versus 46% for the best frontier LLM alone, independently verified by a global MSSP in April 2026. Forty-nine points come from the substrate, not the model.
The practical message from the leaderboard is not "pick a different Claude." It is: do not put a raw LLM in front of your alert queue. Every Claude tops out below 50% coverage, and the gap is structural — not the next-release problem people keep betting on. Skip Opus 4.7 for defensive work. Do not deploy any raw frontier LLM as your SOC. The model is the easiest part of the system to swap. The hard parts — context, retrieval, the investigation loop — are what actually move the score.
Offense and defense are not symmetric problems.
A bigger context window does not fix this. A more capable reasoning chain does not fix this. These are things frontier models already do well, and it was not enough. SOAR cannot fix it either — rule-based playbooks break on the first novel alert and require an engineer to rewrite them. Copilots cannot fix it — they wait for an analyst to ask the right question, and attackers do not respect shift changes.
Simbian's AI SOC Agent runs on the same frontier LLMs that sit on the leaderboard above. The reasoning model is often a top-ranked Claude. What is different is the harness around it — built and pressure-tested through millions of investigation runs in Simbian's Cyber AI Gym:
The result in production: 92% of alerts auto-resolved, 95% verdict accuracy, 100% alert coverage, 24×7×365 with no shift gaps — and on the same Cyber Defense Benchmark used to score the leaderboard, Simbian's harness lifts the best frontier LLM from 46% to 95%, independently verified by a global MSSP in April 2026. Same model. Same data. Forty-nine points of coverage from the harness, not the LLM.
| Approach | Coverage on defense | Fails on |
|---|---|---|
| Raw frontier LLM (best Claude) | 44.5% | unknown ground truth, agent gives up early |
| SOAR playbooks | ~25% automation | every novel alert, constant playbook maintenance |
| LLM copilots | analyst-paced | nights, weekends, anything an analyst forgot to ask |
| Simbian AI SOC Agent | 95% in production | — |
A model is a component. An agent is a product. If you are picking an LLM for your SOC, you are picking the wrong thing. Pick the harness.
→ Book a demo — see Simbian against your own telemetry
Which Claude model is best for cybersecurity? Claude Opus 4.6 is the strongest Claude on the Cyber Defense Benchmark at 44.5% coverage with 7 of 13 MITRE tactics passing. Sonnet 4.6 is the best price-performance pick at 36.9% coverage and $2.21 per investigation. Opus 4.7 ranks last among Claudes at 30.9% with zero tactics passing.
Why does the cyber-safeguarded Opus 4.7 score lowest? Two factors. First, the model optimizes for fast, cheap completion over thorough investigation, with the fastest median wall-clock (243s) and lowest cost ($1.65) of any Claude. Second, Anthropic's Cyber Verification Program safeguards constrain offensive prompts; they do not add defensive reasoning skill. The benchmark scores defensive work, not safety posture.
Can any LLM pass the Cyber Defense Benchmark today? No. Across 14 frontier models from Anthropic, OpenAI, Google, and open-weight providers, none crossed the 50% coverage threshold. The leader sits at 44.5%. By contrast, frontier LLMs routinely score above 80% on offensive-security benchmarks. Defense is the harder problem.
Is Claude the best LLM for SOC work? Yes, by a clear margin. Every Anthropic model on the leaderboard outscores GPT 5.5 (26.4%), GPT 5 (17.4%), and every Gemini and open-weight model. The top four leaderboard slots are all Claude. The best LLM for SOC today is Opus 4.6.
What closes the gap between LLM cybersecurity scores and a real SOC? The harness around the model. On the same benchmark, the best frontier LLM alone scored 46%; the same model wrapped in Simbian's harness scored 95%, independently verified by a global MSSP in April 2026. The 49-point lift comes from context, skills, and the agent loop, not the underlying LLM.
