Loading...
Loading...
Large language models are strong pattern-matchers on code. That is the boring framing. The interesting one is this: an LLM can now sit down with 12,000 lines of kernel source, hold the concurrency model in its head across multiple threads, and point at a subtle memory bug a human reviewer missed. Security researcher Sean Heelan demonstrated exactly that. Using OpenAI's o3 model through its API — no custom scaffolding, no agentic framework, no fine-tuning — he surfaced CVE-2025-37899, a previously unknown zero-day in the Linux kernel's SMB implementation. The bug is a use-after-free that only manifests when two threads on separate connections race against each other. For defenders, the timing is unforgiving. Zero-days ship without a patch, and every hour before the vendor responds is an hour your defence-in-depth strategy is doing the work alone.
Heelan set out to test whether o3 could find a specific class of memory bug — use-after-free (UAF) issues in ksmbd. It did. It also found a second, related bug that wasn't part of the test. That "bonus finding" is the more interesting result. A model that only spots what you tell it to look for is a smarter grep. A model that follows the concurrency graph one function further and flags a related issue is doing something closer to code review.
In this evaluation, o3 outperformed Sonnet 3.7 by roughly 2–3x on the specific benchmark Heelan constructed. The quantitative jump matters, but the qualitative signal matters more. An assistant that can spot critical issues sitting just outside your research focus opens up a different way to work. You stop treating the model as a checklist and start treating it as a second reviewer with unusual patience.
The vulnerability lives in the kernel SMB implementation. When ksmbd handles an SMB2 LOGOFF command, one thread can free an object while another thread — potentially serving a different connection — is still holding a reference to it. There is no synchronization guarding the shared state. The result is memory corruption, and depending on the allocator's behavior, an exploitable condition.
Concurrency bugs like this one are hard for humans because they only exist across thread boundaries. You cannot see them by reading one function. You have to hold the whole state machine in your head, imagine the interleavings, and reason about what breaks when. That is expensive cognitive work. It is also exactly the kind of work LLMs are getting better at, precisely because reading 12,000 lines does not tire them out.
The honest answer is: sometimes, and unreliably. Most static analyzers struggle with cross-thread UAF because the bug lives in the interaction, not in any single function. Dynamic analyzers need a runtime, and getting realistic runtime coverage on a kernel module, a firmware image, or an IoT device is often impractical. AddressSanitizer and its cousins help when you can drive them with representative workloads, but simulating real memory pressure and real concurrency is non-trivial. That is why UAF bugs continue to slip past traditional SAST/DAST pipelines. The tools are not broken. The problem is genuinely hard.
Look at the numbers with a cold eye. o3 caught the vulnerability in 8 of 100 runs. Another 28 runs produced false positives. For a solo researcher chasing a specific hypothesis, those odds are useful — you are trading compute for cognitive effort, and both are cheap compared to a missed CVE. For a vulnerability management program that has to be predictable week over week, those odds are hard to operationalize. You cannot promise your CISO that a critical bug will be caught on the first pass when the base rate says it won't be.
That is not a reason to walk away from the technique. It is a reason to design around it. Run the model many times. Cross-check its output. Feed the false positives back into the prompt as counter-examples. Pair the model with a targeted fuzzing or symbolic execution pass on the functions it flags. The point is not that o3 replaces a human reviewer — it is that a human reviewer paired with o3 covers ground neither can cover alone.
LLMs have real potential as security research assistants. The o3 result is one of the cleanest demonstrations to date. But potential is not production. Turning a promising demo into a reliable pipeline takes the right model for the job, evaluations that mirror the workload, and context that is specific to the codebase and threat model in front of you. Heelan's test shows both sides of that equation: the capability is real, and the noise floor is real.
At Simbian, we run continuous evaluations of frontier and specialist models against the security workloads our agents actually handle. Different tasks reward different models. Simbian's AI VRM Agent and AI CTEM Agent are designed to pair those models with a curated organizational context (Context Lake™) plus our proprietary tuning. That context is the reason a general-purpose reasoning model, on its own, produces the noise we see in the o3 numbers, and a context-aware agent stack produces something you can actually run in a program.
Simbian's AI Agents don't just spot potential vulnerabilities. They classify and prioritize already-detected issues for engineering teams, suggest mitigations, and — when possible — propose test vectors that let you validate defence-in-depth coverage before the vendor ships a patch. The philosophy is self-improving, not self-driving: agents act, humans steer, and the loop tightens with every cycle.
AI plus human researchers, developers, and security professionals is the pairing that actually moves numbers on complex vulnerabilities. A soc program that ignores the model gives up leverage. A program that trusts it blindly gives up predictability. The interesting work sits between those two extremes, and that is where the next generation of vulnerability agents will be built. Stay tuned for a deeper dive into how LLMs are reshaping defensive security workflows.
AI Vulnerability Analysts are not a distant fantasy. Simbian automates vulnerability assessment and prioritization, cuts mitigation time, and gives security teams their attention back. The question isn't whether AI will change how your security teams work. It is how quickly you fold it into how you already work today.
Q: What is CVE-2025-37899? A previously unknown zero-day vulnerability in the Linux kernel's SMB implementation (ksmbd), discovered by security researcher Sean Heelan using OpenAI's o3 model. It is a use-after-free bug triggered by a race condition when handling the SMB2 LOGOFF command across concurrent connections.
Q: How did o3 find the vulnerability? Heelan gave o3 direct access to roughly 12,000 lines of kernel source through the API — no agentic framework, no custom tooling. The model reasoned about thread interleavings and identified both the target UAF class and a second related issue outside the original test scope.
Q: How reliable is LLM-based vulnerability detection today? Mixed. o3 caught the bug in 8 of 100 runs and produced false positives in 28. That signal is useful for individual researchers but hard to operationalize for a predictable vulnerability management program without additional context, evaluation harnesses, and validation steps.
