Research Story
Open-weight DeepSeek nears OpenAI's restricted Daybreak model in Vals network-hacking benchmark
In a Vals benchmark of multi-step attacks on simulated corporate networks, reported by Wired, open-weight DeepSeek completed 73% of objectives on average, against 81% for OpenAI's restricted Daybreak model.
This story was written with the help of artificial intelligence, checked automatically, then reviewed by an editor before publication, under our AI policy.
What Vals tested
Vals, a company that measures AI capabilities, tested how well different models can carry out complex, multi-step network instructions on simulated corporate networks, according to Wired. The benchmark uses two test ranges, Nova and Lighthouse, with 21 and 31 objectives respectively. The overall score averages the two ranges with equal weight, Vals says.
Runs are capped at 500 million cumulative tokens. Vals also says it checked that no model read the answer key, and that each validated objective corresponds to a marker read on a machine that was actually compromised.
Results: a narrow gap
According to Wired, OpenAI's Daybreak model, which is reserved for trusted partners, completed 81% of tasks on average across the two tests, the best result. DeepSeek, an open-weight Chinese model, completed 73% of objectives on average, not far behind.
Vals names the top model as GPT-6 Sol, offered only through the restricted Daybreak programme, with DeepSeek V4.1 Flash 1017 coming just after. On the hardest network, GPT-6 Sol Daybreak completed 27 of 31 objectives in under three hours, after taking administrator control of the first company and using stolen passwords to reach the second, according to Vals.
Vals also states that no model completed the final objectives on two networks, and that only one run across the whole set reached the final stage of the single network.
Cost differences
Vals reports a large cost difference. According to Vals, an attack costs 17 to 46 dollars in tokens with GPT-6-Sol, against 2 to 3 dollars with DeepSeek. Vals also claims DeepSeek matched Daybreak on the first network for under 3 dollars per attempt, against about 22 dollars for Daybreak.
Vals says open-weight models make their late progress thanks to a low cost per token. It adds that Grok 4.7, Kimi K3 and GLM 5.3 Flash stopped well before the end of their budget without making further progress.
What the experts say
Vals says the best closed frontier models are starting to be matched by open models that anyone can download. These are Vals's claims and interpretation, not independent findings.
Rayan Krishnan, Vals's chief executive, told Wired that open models give defenders more incentive to use them, because they can run on their own hardware. Alex Zenla, chief technology officer of Edera, said open models can already produce advanced attacks and that closed labs probably hold more capable unreleased models.
Wired also notes that Anthropic restricted access to Claude Mythos Preview from April through its Project Glasswing programme. OpenAI did not immediately respond to Wired's request for comment.
Open questions
The reported figures come from Vals's simulated networks, and the sources do not say how the results would translate to real-world environments. They also do not detail how Daybreak access is granted beyond describing it as restricted to trusted partners.
| Model | Access | Average objectives completed | Reported token cost per attack |
|---|---|---|---|
| Daybreak (GPT-6 Sol) | Restricted to trusted partners | 81% | 17 to 46 dollars |
| DeepSeek | Open-weight | 73% | 2 to 3 dollars |
FAQ
Which model scored highest in the Vals test?
OpenAI's GPT-6 Sol, available only through the restricted Daybreak programme, scored best, according to Vals. DeepSeek V4.1 Flash 1017 came just after.
How much cheaper is DeepSeek for this kind of attack?
Vals says an attack costs 2 to 3 dollars in tokens with DeepSeek, against 17 to 46 dollars with GPT-6-Sol.
Did any model complete the whole challenge?
No model completed the final objectives on two networks, and only one run reached the final stage of the single network, according to Vals.
Sources
- Free Chinese AI Models Are Getting Scary Good at Hacking, Wired,
- https://www.vals.ai/benchmarks/cyber_offense, vals.ai, (primary source)