Security data & RL environments for frontier models.
Frontier models fail at cyber defense. We supply the verified data, environments, and expert validation to close the gap — built on real attack telemetry, modeled at org scale.
Email our research team directly — click to copy the address.
Defense is the frontier that hasn't been solved.
A breach is a path hidden in a haystack of benign logs — and models go blind exactly where it matters.
Offense is mostly solved
Deductive and recall-heavy — exploits, code, CTFs. There is abundant signal on the web, and models already score well.
Defense is different in kind
Abductive: reconstruct an unknown attacker's intent from a haystack of benign-looking logs. Almost none of this reasoning exists in pretraining.
It's a data + environment problem
You can't read defensive reasoning off the web, and you can't crowdsource it. It has to be grown with RL against verifiable rewards — which needs an environment that can score an investigation as right or wrong.
The category is wide open.
frontier models pass the Cyber Defense Benchmark
best model's MITRE ATT&CK coverage — the passing bar is 50%
MITRE ATT&CK tactics the benchmark measures, end to end
The shape collapses on the tactics that define a breach — lateral movement, credential access, exfiltration.
Every bar; not one crosses the 50% line.
Unlike offensive benchmarks that saturate in months, this one stays unsolved — every model goes blind on the tactics that define a real breach: lateral movement, credential access, exfiltration.
Two ways to work with us.
Data provider
Verified security training data, the Holodeck RL environment, and the Cyber Defense Benchmark. You train on it.
Human validation
Our network of career SecOps analysts grades your tasks and model outputs — the judgment calls a machine can't verify.
Three ways to make a model defend.
Verified security trajectories
Real investigations and attack chains, decomposed into gradeable tasks across threat hunting, detection engineering, triage, and offensive security. Every unit ships with verified ground truth — machine-checkable wherever the task allows.
See the task taxonomy →Holodeck
A deterministic, Gymnasium-compatible gym where a model learns to hunt and is scored on machine truth — verifiable rewards, no reward-model drift.
How Holodeck works →The Cyber Defense Benchmark
The published eval — designed to resist memorization — that shows where models fail, usable as your north-star security metric.
Read the benchmark →Every use case decomposes into gradeable task categories.
The unit we build data and evals around isn't a use case — it's each discrete skill inside it.
Threat Hunting
- hypothesis generation
- query generation
- evidence correlation
- lead / pivot expansion
- attack attribution
- timeline reconstruction
Detection Engineering
- detection authoring
- coverage-gap analysis
- false-positive tuning
- rule validation
- data-source mapping
- detection-as-code review
Triage & Investigation
- alert enrichment
- prioritization
- root-cause analysis
- disposition (TP / FP)
- response actions
- escalation
Offensive / Pentest
- recon
- exploitation
- privilege escalation
- lateral movement
- impact
- reporting
Each category is scored independently — so we can produce targeted data and measure a model skill by skill. Extends to tool-use / integration trajectories — agents driving security connectors — for agentic data, not just text.
Holodeck: a deterministic gym that scores hunting against machine truth.
Any base model drops in
A Gymnasium-compatible interface: observe (threat briefing + prior results) → act (query / submit / give-up) → reward. No bespoke harness to build.
Verifiable reward
In Holodeck, ground truth is deterministic — the reward is exact and unfakeable, with no reward model to drift and no LLM judge to game.
Real attacker behavior, modeled at org scale
Deterministic replay with seeded mutation; same seed → identical run. Reproducible and reusable, like a versioned environment artifact.
Compatible with the RLVR toolchain your team already uses — tasks, tools, and reward functions as first-class, verifiable artifacts.
Where judgment is required, bring security experts — not crowd labels.
Our network of career SecOps analysts — threat hunters, detection engineers, and incident responders with decades in the field — manually reviews and grades tasks and model outputs.
Grade model outputs on real security tasks
Scored by people who do this for a living.
Preference & RLHF data
Expert rankings of competing outputs, for reward modeling.
Judgment-task ground truth
The calls that can't be mechanically verified: severity, escalation, reasoning quality.
Held-out human validation
An independent, human-graded set to catch reward hacking (anti-Goodhart).
Holodeck automates the verifiable reward. Human validation covers what it can't. Priced hourly — deep security expertise, not general crowd work.
Generalist data vendors staff doctors, lawyers, and bankers. We staff the people who actually run security operations.
You can't crowdsource security data. We build it.
Grounded in real operations
Our attack and organization models are built on real attacker telemetry — grounded in operating security through the world's largest MDRs, not fabricated from scratch and not crowd-labeled.
Verified & clean to license
Every trajectory carries verified ground truth; the artifact is our own IP. The moat is the modeling and the verification — not hoarded data.
Built by people who've shipped security and systems
Founders from Fortanix, NVIDIA, and Twitter; authors of the Cyber Defense Benchmark.
Built to slot into post-training.
Questions from research teams
Bring a base model. We'll measure the lift together.
A bounded first experiment: baseline on the Cyber Defense Benchmark, train in Holodeck, measure the delta.