27JUL

Tom's Hardware challenges Mythos zero-day claims

2 min read

10:02UTC

A technical review found Anthropic's marketing relied on 198 manual reviews to support claims of thousands of severe vulnerabilities.

← Back to SAP freezes R&D headcount as others deny AI Jump to analysis ↓

EconomicDeveloping

Key takeaway

Only 198 manual reviews support Anthropic's claim of thousands of zero-day discoveries.

Tom's Hardware published a critical review of Anthropic's Mythos claims on 9 April, noting that the "thousands of zero-days" assertion rested on only 198 manual reviews ¹. Many of the flagged vulnerabilities were in outdated software no longer in active use. The gap between Anthropic's marketing language and the verified sample is wide enough to warrant caution.

The Bessent-Powell emergency meeting at Treasury headquarters proceeded regardless of this scrutiny. Challenger data confirmed AI-attributed cuts crossed 107,094 the same month , suggesting federal regulators assessed the systemic risk of AI broadly, beyond Mythos's specific claims. Whether Mythos found hundreds or thousands of exploitable flaws, the CyberGym benchmark score of 83.1% versus 66.6% for its predecessor represents a measurable capability jump that the twelve Glasswing partners will deploy in production environments.

Deep Analysis

In plain English

When Anthropic announced that Claude Mythos had found 'thousands' of serious security flaws in software, it was a dramatic claim. Tom's Hardware, a technology publication, looked at how Anthropic had actually counted those flaws. The answer was: 198 human reviewers manually checked the model's outputs. Many of the flaws it identified were in old software that organisations had already stopped using. The gap between 'thousands of vulnerabilities' and 198 verified reviews is significant. The US Treasury and Federal Reserve held their emergency meeting with bank CEOs regardless of this critique, which suggests the regulators assessed the risk from the model's overall capability trajectory, not just the specific zero-day count.

Source Landscape

This story draws on neutral-leaning sources

Primary parallel: The 2016 controversy over DeepMind's AlphaGo claims, in which Google's marketing described the system as defeating the world champion 'under official match conditions' when several conditions differed from tournament rules. Independent verification eventually confirmed the underlying capability while correcting specific claim framing. The Mythos methodology question follows the same pattern: contested marketing language around a genuine capability advance.

Counter-parallel: IBM's 1997 Deep Blue versus Kasparov match produced verified, independently adjudicated results that settled the capability claim definitively. Cybersecurity benchmarks lack equivalent independent adjudication, making the gap between Anthropic's 198-review sample and its 'thousands' headline durable in ways that chess results were not.

Consensus view: Trail of Bits co-founder Dan Guido and Veracode's Chris Wysopal assess the 198-review gap as a credibility problem for Anthropic's marketing rather than a refutation of the underlying capability: the CyberGym benchmark improvement from 66.6% to 83.1% is independently verifiable, and that gap is operationally significant regardless of zero-day count methodology.

Counter-view: Recorded Future's intelligence analysts note that threat actors routinely inflate capability claims to generate deterrence value, and that AI vendors have structural incentives to do the same. Overstated AI security capabilities may cause organisations to misallocate defensive resources toward theoretical AI attack vectors while under-investing in conventional vulnerability management.

Key tension: Whether the verified CyberGym benchmark gain represents a genuine operational security threat or a marketing projection requiring independent validation before driving regulatory and corporate resource allocation.

Sources:Tom's Hardware

Mentions:Google →US Treasury →CyberGym →Federal Reserve →Prima →Anthropic →Claude Mythos Preview →

First Reported In

Update #5 · The model they won't release

Tom's Hardware· 10 Apr 2026

Read original →

Causes and effects

This Event

Tom's Hardware challenges Mythos zero-day claims

Independent scrutiny of Mythos's capability claims introduces uncertainty about the model's actual security impact, even as regulators acted on the headline numbers.

Led to

AISI confirms Mythos 20-hour attack chain

AISI partly vindicates Tom's Hardware's critique of single-task superiority claims while confirming a distinct attack-chaining capability the Hardware review did not assess.

Occurred 15 Apr 2026

Read story →

Different Perspectives

European Commission

The European Commission's draft Annex III guidelines, closed for comment on 23 July, treat algorithmic scoring in recruitment, pay and termination as high-risk regardless of whether a human signs off, echoing Spain's Audiencia Nacional ruling 101/2026 on concealed scheduling algorithms. Brussels is shifting the fight from counting AI job losses to assigning legal liability for the tools themselves.

Office for National Statistics

The Office for National Statistics recorded UK vacancies rising to 712,000 on 21 July, the first quarterly increase this beat has tracked, with payrolled employment down 85,000 on the year against May's 210,000 fall. The bulletin names no AI cause anywhere, and that is the point: nothing in the release confirms the displacement story it gets cited to support.

Christian Klein, SAP

Christian Klein told investors on 23 July that SAP's research headcount will not grow for twelve months because AI agents and their token costs are absorbing the work, not because SAP is cutting jobs. He frames it as commercial arithmetic: the cost of AI-assisted coding tokens plus the salaries specialist AI hires command, not people being replaced by machines.

Betsey Stevenson, University of Michigan

Betsey Stevenson argued that the 187,000 jobless-claims reading describes a market that hires little and fires little, not one AI is emptying. She said the real damage hides in eligibility rules and suppressed job postings, not in the headline layoff counts employers keep denying.

Comisiones Obreras, UGT and Concentrix's A Coruña works committee

Comisiones Obreras, UGT and Concentrix's A Coruña works committee blamed Microsoft's push toward AI self-service for the 80 redundancies unions signed off on 22 July, not unavoidable business cause. A second Coruña procedure covering 80 more jobs runs to a 31 August deadline, and the unions want the state, not the employer, setting the pace of AI-driven cuts.

Stanford's 'We Must Act Now' signatories

More than 200 academics, including 16 Nobel laureates, published a 13 July letter warning of AI-driven labour disruption, citing Daron Acemoglu's NBER estimate that AI's total factor productivity gain stays under 0.66% over ten years. The letter's own cited economics sit well below Goldman Sachs Research's 1.5-percentage-point estimate published the same week.