Loading...
Loading...
Qwen 3.8 27B cybersecurity performance inverts its launch-week reputation. The 27-billion-parameter local LLM runs on a single 80 GB GPU and scores 52 on the Artificial Analysis Intelligence Index, matching hosted GPT-5.6 Luna. On Simbian's Cyber Defense Benchmark, hunting real Windows attack logs in August 2026, it finds 10.6% of the attack to Luna's 15.7% and bills $0.24 a hunt against Luna's $0.08.
Alibaba shipped Qwen 3.8 27B on August 14, 2026, and every Qwen 3.8 27B cybersecurity take that followed reached for the same comparison: GPT-5.6 Luna. On the scoreboard they were reading, that comparison is earned. Both models score 52 on the Artificial Analysis Intelligence Index, and only one of them fits on a graphics card you can buy.
We read a different scoreboard.
▶ Watch the webinar — Why LLMs Fail in the SOC A 30-minute walk through the Cyber Defense Benchmark: how every frontier model is scored, the per-tactic breakdowns, and why no LLM clears 50% coverage on real attack logs.
Security teams run a local LLM for reasons that have nothing to do with accuracy: data that never leaves the building, a model that runs on an air-gapped network, and an inference bill nobody else gets to re-price. Every conversation about a private LLM in a SOC starts in one of those three places, and most of them start on a Llama or Mistral checkpoint because that is what the local tooling grew up around.
Qwen 3.8 27B answers all three unusually well. It is dense, Apache 2.0 licensed, and reads images and video natively, which matters when the evidence in a ticket is a screenshot, and its 256,000-token context window decides how much of a log window fits in a single pass.
The hardware is the part that changes the conversation. At full BF16 precision the weights want roughly 54 GB, so one 80 GB card runs it. Quantized to four bits it drops to a 17 GB GGUF file that fits a 24 GB desktop card under Ollama, LM Studio, llama.cpp, or vLLM.
That is a security team's whole inference stack, on one machine, for the price of a workstation.
So the procurement question stops being can we run our own model and becomes what do we give up when we do. For this model, nobody had put a number on it.
Artificial Analysis rates Qwen 3.8 27B (xhigh) at 52 on its Intelligence Index, the same score it gives GPT-5.6 Luna (max). Simon Willison, writing on August 16, put the appeal plainly. A 17 GB file doing this on a home machine is, in his word, "a miracle."
He also wrote the more useful sentence. "Qwen 3.8 27B is excellent, but it defaults to wildly overthinking things." In one run, a pelican-on-a-bicycle image took 21 minutes and 22,276 reasoning tokens to produce 3,223 tokens of output. Seven tokens of thinking for every token delivered.
His fix is to turn the reasoning down. "Run Qwen 3.8 27B on low or even no reasoning levels at first," he writes. Note what that costs you. That 52 is the xhigh number. So is ours. Turn the reasoning down and you are no longer running the model that tied Luna.
On the Cyber Defense Benchmark (arXiv 2604.19533), Qwen 3.8 27B reconstructs 10.6% of a full attack chain. The benchmark drops an agent into more than 100,000 real Windows security events with no alert, no question, and no hint, and scores how much of a genuine multi-stage intrusion it can pull back out through iterative SQL queries. Ground truth is deterministic, so nothing is graded by another model.
Coverage is the fraction of malicious events the agent finds and submits. Not spots. Submits.
Local LLM coverage and cost per hunt on the Cyber Defense Benchmark, August 2026:
| Model | Coverage | Median cost per hunt | Weights |
|---|---|---|---|
| DeepSeek 4 Flash | 29.7% | $0.17 | Open |
| Qwen 3.8 Max | 29.5% | $1.59 | Open, custom license |
| Qwen 3.7 | 19.1% | $0.21 | Open |
| Gemini 3.6 Flash | 18.7% | $1.34 | Hosted |
| GPT-5.6 Luna | 15.7% | $0.08 | Hosted |
| Qwen 3.8 27B | 10.6% | $0.24 | Open, Apache 2.0 |
| Nemotron 3 Super | 9.1% | $0.15 | Open |
Qwen 3.8 27B was run after the current public leaderboard snapshot of August 17, 2026, so its row publishes with the next refresh. Every other row above is live on the benchmark page today.
GPT-5.6 Luna, tied on intelligence, finds about 48% more of the attack for a third of the price. DeepSeek 4 Flash, also open, finds nearly three times as much for less money than the model you would have self-hosted.
Qwen 3.8 27B does clear Nemotron 3 Super, which ships at 120 billion parameters. That comparison flatters the smaller model more than it should. Nemotron is a mixture-of-experts design that activates only 12 billion parameters per token (Qwen's 27 billion are all dense, all the time). On the compute that actually runs, Qwen is the bigger model of the two. It wins by 1.5 points.
Across 25 models the leader, Opus 5, reaches 45.1% and clears the 50% bar on seven of 13 MITRE ATT&CK tactics.
Nothing passes. Defense is harder than offense and draws a fraction of the post-training attention, so the benchmark sits nowhere near saturation.
No. A local LLM costs more at this size. Qwen 3.8 27B bills $0.24 a hunt against GPT-5.6 Luna's $0.08, and almost all of that gap is the list price, not the talking.
Qwen 3.8 27B lists at $0.50 per million input tokens and $3.00 per million output, against Luna's $0.20 and $1.20 — 2.5 times the rate on both sides of the ledger before either model writes a word. Our observed gap is 3.0 times. That leaves roughly 1.2 times as much token burn to explain.
Which is the part worth dwelling on, because the launch-week story was verbosity. Artificial Analysis clocked this model at 160 million output tokens against a 43 million median for open-weight models of similar size, roughly 3.7 times the typical burn. If that reproduced on a hunt, a hunt would cost nine times Luna's. It costs three.
Self-hosting Qwen 3.8 27B cost us about $1.08 per investigation, four times its hosted price, which undercuts the main reason for running it yourself. The same hunts landed there largely because the prompt caching a hosted endpoint handles for you is awkward to reproduce on a local serving stack. That figure isn't benchmark data and we haven't published a methodology for it, so treat it as a field observation — but it points the wrong way for anyone self-hosting to save money.
What we cannot tell you from this data. Whether verbosity also explains the low coverage. The token counts come from a bounded reasoning index; the Cyber Defense Benchmark records coverage, cost, and investigation time, never tokens per query. The simpler explanation is that 27 billion dense parameters are too few for an investigation running hundreds of turns. Both the 52 and our run used the xhigh reasoning setting Willison tells you to avoid, and we have not measured what low reasoning does to either number. TrustedSec saw the same shape on the offensive side, clocking
qwen3:32b— an earlier, larger Qwen, not the 27B — at 16,924 tokens a run against a smaller model's 2,651, for the worst pass rate in its set. Theirs was the biggest model in its set and ours is the smallest, so size cuts both ways.
Qwen 3.8 27B sits on the wrong side of the cost-coverage frontier, and it is the only model in this cohort that does. Across the full 25-model leaderboard the relationship runs opposite to the intuition: the six highest-coverage models all cost more than $1.50 a hunt, and the five lowest all cost under $0.35. Spending tokens is not the failure. Spending them and finding nothing is.
It pays mid-tier token bills for bottom-quartile coverage.
Yes. Inside the Qwen family, coverage nearly triples from 10.6% on the 27B to 29.5% on the 2.4T Max, close to linear with size. The two small models are priced within a few cents of each other, so at the bottom of the range the money buys you nothing and the parameters buy you everything.
Open source LLM coverage by model size, Qwen family:
| Qwen model | Coverage | Median cost per hunt |
|---|---|---|
| Qwen 3.8 27B (27B dense, Apache 2.0) | 10.6% | $0.24 |
| Qwen 3.7 (size not disclosed) | 19.1% | $0.21 |
| Qwen 3.8 Max (2.4T MoE, 95B active) | 29.5% | $1.59 |
Roughly the same money buys 10.6% or 19.1%, depending only on which model you point at the logs. Spend about seven times more per hunt than the 27B and coverage nearly triples. Within one family, on one task, scale tracks coverage more cleanly than any token-efficiency story does. Which is worth holding onto before anyone blames overthinking: the 27B is the smallest thing on this table, and small explains it on its own.
Qwen 3.8 27B ships under Apache 2.0. Qwen 3.8 Max ships under a custom license that requires a separate agreement above $50M in annual revenue, and its open checkpoint is text-only, without the vision and long-context features of the API build we benchmarked. If procurement is why you're reading this, those are two different products wearing one word.
A local LLM is good for the SOC work a human still grades: log summarization, detection drafting, indicator enrichment, and query generation. It is poor at the open-ended hunting nobody checks.
A 256,000-token window is what makes those jobs practical on one machine: summarizing a large log dump without shipping it off-prem, drafting detection logic and explaining what a rule will and will not catch, enriching an indicator against internal-only data, or turning an analyst's question into a query that a person then runs and reads.
Every one of those has a human checking the answer. Open-ended hunting is the job where nobody does. The model decides when it has looked hard enough, and the only signal that it stopped early is the intrusion you never hear about. That is the job where 10.6% and 29.7% are different answers to the same question.
The mistake is treating model size as the way to buy sovereignty. Data residency and air-gapped operation are properties of where the model runs, and you can have them without dropping from 29.7% coverage to 10.6%.
Most of what closes that gap lives outside the weights: the skills that encode how your senior analyst investigates, the memory of what your environment normally looks like, the planner that decides which query comes next, and the critic that refuses to let an agent stop early. That is what an LLM harness does.
Our AI Threat Hunt Agent is built as one, which is why pointing it at a model endpoint inside your own boundary costs you nothing in sovereignty. It won't turn a 10.6% model into a 45% model, though. Model choice still matters. Pick it on defensive data.
The sovereignty case for running your own model survives this data intact.
The cost case doesn't. Buy the boundary. The discount isn't included.
Q: What is a local LLM? A local LLM is a language model whose weights you download and run on hardware you control, such as a workstation GPU, an on-prem server, or an air-gapped cluster. No hosted API call, and nothing leaves your network. Open-weight releases like Qwen 3.8 27B, published under Apache 2.0 on Hugging Face, are what make this practical at 27 billion parameters and roughly 54 GB at BF16.
Q: Can you run Qwen 3.8 27B for security operations? Yes, and Qwen 3.8 27B makes it more practical than it has ever been. It runs on a single 80 GB card at full precision, or a 24 GB card as a 17 GB four-bit GGUF, under an Apache 2.0 license. It runs. The open question is how much of an attack it finds once it does. On the Cyber Defense Benchmark this model reconstructs 10.6% of a full attack chain.
Q: What is the best local LLM for cybersecurity if it must run on-prem? DeepSeek 4 Flash. On the Cyber Defense Benchmark it leads the open-weight launch cohort at 29.7% coverage for a median $0.17 per investigation. Qwen 3.8 Max reaches 29.5% but costs roughly nine times more per hunt, and its open checkpoint carries a revenue-gated custom license where the 27B carries Apache 2.0. Across the full 25-model leaderboard, GLM 5.2 is the strongest open-weight model at 35.0%. No open-weight model, and no hosted model either, clears the 50% coverage bar on every MITRE tactic.
Q: How much does it cost to self-host an LLM? More than the API price suggests. Qwen 3.8 27B bills $0.24 per investigation on hosted pricing, but on our own self-hosted runs the same hunts landed nearer $1.08, because the prompt caching a hosted endpoint handles for you is awkward to reproduce on a local serving stack. Treat that figure as a field observation from our runs, unpublished and without a methodology behind it.
Q: How do you run Qwen 3.8 27B locally with Ollama? Download the weights and serve them with a local runtime. Ollama and LM Studio are the easiest starting points, llama.cpp gives you the most control over quantization, and vLLM is the usual choice when you need throughput for an agent workload. Weights come from Hugging Face, and a four-bit GGUF of a 27B model is about 17 GB on disk.
Q: What hardware do you need to run Qwen 3.8 27B? A 24 GB consumer card is the floor and an 80 GB card is comfortable. Full BF16 precision wants roughly 54 GB, so that is H100 territory. FP8 serving brings the weights to about 27 GB, and four-bit quantization to a 17 GB GGUF that fits a 24 GB desktop card, though long-context work needs headroom above the weights for the KV cache. Above 80 GB you are into multi-GPU, where the sovereignty argument starts costing real money.
Q: Is a local LLM actually private? Yes for data residency, and that is the part worth paying for. The weights run on your hardware, the logs never leave your network, and no vendor sees the prompt. What you don't get for free is accuracy. On the Cyber Defense Benchmark Qwen 3.8 27B found 10.6% of an attack where GPT-5.6 Luna, hosted at the same intelligence score, found 15.7% — though other open-weight models in the same cohort beat both. Privacy is a deployment property you buy separately from coverage.
Q: Can a local LLM replace ChatGPT or Claude for security work? Not on the evidence here. Every model tested fails the Cyber Defense Benchmark, hosted and local alike, so this is not a local-versus-hosted argument. The gap that matters is between a bare model and a model inside a harness that plans queries, carries environment memory, and refuses to stop early. Pick the model on defensive data, then run it wherever your compliance boundary requires.
Q: Is Qwen 3.8 27B good for cybersecurity? No. Qwen 3.8 27B and GPT-5.6 Luna score identically at 52, and Luna finds about 48% more of a real attack for a third of the cost. General indices measure reasoning, knowledge, mathematics, and coding on bounded problems with clean success signals. Defensive work is unbounded, the volume of malicious activity is unknown at the start, and normal traffic overlaps heavily with malicious traffic.
Q: Is an open source LLM as good as a hosted model for threat hunting? Sometimes better. DeepSeek 4 Flash at 29.7% beats every hosted model costing under $1.50 a hunt, and open-weight GLM 5.2 sits at seventh of 25 overall. Open weights predict nothing about defensive performance either way, which is why each model has to be measured on the job it'll actually do.
Q: Why does Qwen 3.8 27B cost more per investigation than GPT-5.6 Luna? Mostly list price, not verbosity. Qwen 3.8 27B costs $0.50 per million input tokens and $3.00 per million output against Luna's $0.20 and $1.20 — 2.5 times the rate before either model writes a word. The observed gap on a hunt is 3.0 times, so token volume accounts for only about 1.2 times on top. Artificial Analysis measures it at 3.7 times the median burn on their own index, but that does not reproduce on our invoice.
