OSS Scanner skips human review: Anthropic expects over 90% true, but prints no rate for the 4,824 already sent

More than 90 in 100 reports true is an expectation, and 4,824 reports have already gone out unchecked

On 8 October 2026 Anthropic launched OSS Scanner, a free service for open source maintainers who opt in. Its announcement says the reports are "model-generated and sent without human review", and that Anthropic expects "a true-positive rate above 90%". Read as arithmetic that is short (derived): fewer than 10 reports in every 100, and fewer than 100 in every 1,000, would fall short of true. Each lands with a maintainer who is, on our reading, the first person to look at it.

The record to test the expectation against is Anthropic's own disclosure dashboard, last updated at 19:47 UTC on 2 October. It counts 29,439 candidate findings. Outside security firms reviewed 6,123 of them (20.8 per cent, derived) and confirmed 5,674, which is the 92.7 per cent the page prints. Of the 6,157 findings reported to maintainers, 4,824 (78.3 per cent, derived) went direct, without that independent check, under the label "may contain false positives". The page prints no accuracy rate for those 4,824. And 516 findings, 8.4 per cent of those reported (derived), are marked "patched upstream".

That does not establish that unreviewed reports will be 90 per cent right. Here is what it leaves open.

• That the 92.7 per cent carries over. It is the share of reviewed candidates the firms confirmed. By Anthropic's glossary it counts duplicates and "won't fix" findings as true positives, and it was measured on the 6,123 sent for review, not on the 23,316 that were not (derived). The announcement does not say which definition "above 90%" uses.
• That maintainers can absorb the volume. The same dashboard says its acknowledgement count "reflects the fact the OSS maintainers are experiencing a higher volume of inbound security findings".
• That anything was fixed faster. 516 patches against 6,157 reports is a count, not a speed. The announcement says that in Glasswing "we often saw months pass between a vulnerability being found and being fixed".
• That operators of power, water or transport are better protected. The second initiative, the Critical Infrastructure Defense Program, arrives with no result, no count and no named operator.

Anthropic sells the models that run this scan, including a paid code-scanning product, Claude Security, and the Defender Advantage Fund that keeps OSS Scanner free is made up of 35 million dollars in Claude credits, so it has a commercial interest in showing that its models find real bugs. Everything below is attributed to a page we read, and where we infer, we say so.

What OSS Scanner states, and what it leaves out

Anthropic's pages for the service are a news post, a research post, an FAQ, an agreement and a repository README. Together they answer most operational questions. The tables set each answer beside its silence.

How OSS Scanner works, from Anthropic's news post, research post, FAQ, agreement and repository README, read 9 October 2026

Question	Stated	Not stated
Who can join	Core maintainers of projects with "a critical impact on infrastructure and user security", on OSS-Fuzz-like criteria, case by case. Anthropic confirms core-maintainer status. Enrolment is a pull request to a public repository.	How many projects it will take. Anthropic says it "may adjust the acceptance criteria over time".
What a report holds	A description, a reproducer (the news post says "proof of concept"), a proposed patch where available, and a severity. The research post adds a bisection to when the bug arrived, "where possible".	How detailed or contained the reproducer is. How many errors the double-checking agents remove.
Who receives it	One primary contact by email, optional copies, or PGP to the primary contact only. The addresses in the enrolment file are public. Reports are Anthropic's "confidential material", to be shared only with authorised maintainers.	Any hand-off to a coordinator or to downstream users.
Cost	Free: "We cover the full cost." The Defender Advantage Fund "keeps OSS Scanner free".	The cost of maintainer time, or of the offline build the project must supply. Anthropic may "modify, suspend, or end the Service" at any time.
Volume	Scans are "periodic". Frequency "may depend on the number of projects in our pipeline, how widely it's used, and other factors". Early trials produced hundreds of reports across dozens of projects.	Any expected, average or maximum count per project, or any cap.

What the maintainer controls, from the same pages

Question	Stated	Not stated
When a report is wrong	The agreement says reports "may be incomplete and inaccurate", may misjudge severity and may propose patches that "impair or break functionality". The participant is "responsible for reviewing each Report". Provided "as is"; Anthropic's liability is capped at $1,000. Feedback is by replying to the email.	Any correction, withdrawal or duplicate-handling process, or a time to respond.
Reject or limit	Pause by setting disabled to true, or leave by deleting the project directory, each by pull request. An optional threat-model file can say what to test, how to rate severity and how reports and patches should look.	A cap or schedule the maintainer sets. How fast Anthropic merges a pause: the README says it reviews and merges enrolment pull requests.
Disclosure	No 90 day period on unvalidated reports, and the README says Anthropic "will not make them public". If Anthropic later validates one by hand it may disclose 90 days after telling the maintainer. It may apply a period to some reports later, with notice.	What would trigger a period, how long the notice is, or how a maintainer learns a report was validated. The FAQ promises an opt-out; the agreement says "reasonable advance notice".
Severity	Each report carries a severity, and Anthropic warns some will be wrong. A maintainer can state a severity rubric in the threat-model file.	A measured accuracy for the scanner's severity. The nearest figure is for Claude's first ratings in the earlier process (below).

Who has joined is not stated either. At about 09:51 BST on 9 October, and unchanged at 10:07, the public repository, created at 17:25 UTC on 8 October, listed 96 pull requests opened since 19:52 UTC that evening: 83 open, 13 closed, none merged, and a single commit, the initial one. We did not read or classify the requests and do not claim they are enrolments. The README says Anthropic merges only enrolment pull requests and checks that the author is a core maintainer first. The count says only that, about thirteen hours after the first request, none had been accepted (derived).

A wrinkle in who is covered. Projects that do not opt in keep receiving human-verified disclosures. But the dashboard says direct, unreviewed disclosure also happens for "other vulnerabilities", not only when maintainers ask, while the research post puts the "nearly 5,000" down to maintainers asking for everything. The pages do not say how the 4,824 split between the two.

Anthropic's own early sample is 88 or 99 per cent, depending on the definition

The research post holds the only scanner-specific accuracy test. Anthropic asked the expert penetration testers who review its human-verified disclosures to check 97 critical and high severity findings from an early version, across 48 projects. Of the 97, 85 "met the bar for our CVD process". Of the other 12, 11 were real but duplicated known issues or other findings from the same scan, and one was invalid.

Anthropic's early sample, 97 critical and high findings from 48 projects, read two ways. Research post of 8 October 2026; shares and per-100 figures derived by us

Reading	Count and share	Per 100 reports
Met the bar for human-verified disclosure	85 of 97, 87.6%	12 fall short
Real, duplicates included (the dashboard's definition of true positive)	96 of 97, 99.0%	1 invalid
Anthropic's stated expectation	Above 90%	Fewer than 10 fall short

On the strict reading the early sample sits below the expectation, and on the dashboard's definition it sits far above. The announcement does not say which one "above 90%" means, and the gap is 12 reports against 1 in every 100 (derived). A duplicate is a true positive to Anthropic, but it is still a message a maintainer has to open, match and close.

The sample is narrow in ways the post does not hide: critical and high findings only, an early version of the pipeline, testers Anthropic engages, and no statement of how the 97 were chosen from the whole. The post adds that maintainers "have seldom told us a high or critical finding was invalid", and that some say "severity ratings can be inflated or the scanner misunderstood the project's threat model". Of four project testimonials, one reports 74 reports received, all but two valid and five becoming CVEs (97.3 and 6.8 per cent, derived). That is a vendor's pick of early users, and the only per-project volume on the pages.

The severity label is the weakest field, on Anthropic's own dashboard

The dashboard compares Claude's first severity rating with two later ones. Against the outside firms' (n = 1,337) it prints 83.0 per cent exact agreement and 97.1 per cent within one band. Anthropic's explanation is that its ratings "are produced before any maintainer input", that maintainers "often apply project-specific severity rules that Claude does not have access to at run time", and that this is why the firms' ratings "tend to be lower".

Claude's first severity rating against later assessments, from the dashboard of 2 October 2026 and its machine-readable record. Shares derived by us; the maintainer comparison is not charted on the page

Comparison	Count	Reading
Claude against the firms, exact match	1,110 of 1,337	83.0%, as printed
Claude "critical", firm also "critical"	116 of 259	44.8% (derived); 121 were rated high and 22 lower
Claude against the maintainer, exact match	75 of 163	46.0% (derived); Claude higher in 76, lower in 12
Maintainer-assessed findings as a share of the 6,157 reported	163	2.6% (derived); only where a maintainer gave a severity

Two cautions. These are Claude's first ratings in the earlier process, not the scanner's, and the pages do not say the scanner rates better. The maintainer sample is also small and self-selecting. It does show the direction of error: the model's first rating was higher than the maintainer's in 76 of 163 cases and lower in 12 (derived). The scanner's answer is the optional threat-model file, which helps only the maintainer who writes one. The label still travels: it is the figure a maintainer sees first and a downstream consumer is most likely to inherit.

The bottleneck moves to the maintainer

Anthropic's framing is candid, and it is the centre of this story. The announcement says it is "easier than ever to find vulnerabilities, but verifying, prioritizing, and fixing these findings remains challenging", and that Glasswing has not yet produced "a sufficient reduction in cyber risk". The research post says "we remain bottlenecked on our human capacity to validate these findings".

The dashboard shows what that looks like. Of the 5,674 findings the firms confirmed, 1,333 had been reported after review, so up to 4,341 were waiting (derived). That figure includes duplicates and "won't fix" cases, but the page itself says many confirmed bugs are unreported "due to capacity limitations". Meanwhile the unreviewed, direct reports rose from 1,129 on 22 May, in Anthropic's earlier results post, to 4,824 on 2 October: 4.3 times as many, about 28 a day over 133 days (derived).

OSS Scanner turns that second route into a standing service. Our reading, and it is inference: the step Anthropic calls rate limiting does not disappear. It moves from outside firms that Anthropic engages to the maintainer, whom the pages do not say it pays. The FAQ says the service is for projects that can keep up, and also that many are overwhelmed. And the test it applies, "critical impact on infrastructure and user security", measures how much a project is depended on, not how many people maintain it.

Anthropic's disclosure policy, last updated on 6 March 2026, says "Every report we send generally reflects a finding that a human security researcher has reviewed and confirmed". It promises to "pace our submissions to what maintainers can actually absorb" and not to "submit large volumes of findings to a single project without first reaching out". On our reading, OSS Scanner keeps the second promise by consent: a maintainer who opts in has agreed. It sets no pace, and the FAQ ties scan frequency to Anthropic's pipeline rather than the maintainer's capacity.

The case for the service is not weak. The post quotes early users who found reports "as good and sometimes better than what we get from people", and says a reproducer lets a maintainer "verify it right away". On those terms the unreviewed route trades Anthropic's reviewer time for the maintainer's, at a price the maintainer may judge worth paying. The price is theirs to work out before opting in, and the pages give them no per-project volume to work it out with.

Where the human review sits in each route

The diagram draws only what Anthropic's pages describe. The one step that is our reading, not its words, is the last: that with no review before delivery, the first human to read a scanner report is the maintainer.

The Critical Infrastructure Defense Program: eleven partners, a promise and no results

The announcement describes the programme as bringing "frontier Claude models, on-site engineers, and our threat research" to the providers that operators of power grids, water systems, factories and transport networks rely on. Its eleven founding partners are Accenture, Booz Allen, CrowdStrike, Deloitte, Dragos, Hitachi, Insane Cyber, Nozomi Networks, Palo Alto Networks, PwC and Rockwell Automation. Anthropic groups them as consulting and technology firms, security companies and the manufacturers that build and patch equipment. The announcement carries a statement from an executive at each; they are endorsements, and we print none of them.

The Critical Infrastructure Defense Program as described in Anthropic's announcement of 8 October 2026 and its interest-form page, read 9 October

Question	Stated	Not stated
Which models	"Frontier Claude models".	Which ones. The Cyber Verification Program page of 6 October names three models across its tiers; nothing says this programme uses those tiers or their controls.
On-site engineers	"On-site engineers", and "Anthropic engineering talent".	What they do, where they sit, who vets them, or whether any works inside an operator's network.
Data	Nothing on the pages read.	What partners or their customers share with Anthropic, how long it is kept, or where it is processed.
Who takes part	Eleven founding partners and "a small cohort of providers".	Any operator of a power, water or transport network as a participant. None is named.
What is evidenced	"Several partners are currently working with Claude to fix vulnerabilities and help customers do the same."	How many of the eleven, how many vulnerabilities, in which products, whether any fix reached a running system, or any date.
Terms and growth	More partners and sectors "over the coming months". Companies that build security products or services for critical infrastructure can register interest.	Eligibility criteria, who pays for the models (SiliconANGLE reports Axios raising this), or a date. The form page's own description says 2027; the announcement says "coming months".

Anthropic is open about the limit. It says "Critical infrastructure is hard to defend in many ways that AI cannot fix", that in operational technology "a fix may have to wait until it can be applied safely to running machinery", and that "in some rare cases, this might take decades". That is the honest boundary of the idea for a UK operator: a faster finding does not shorten a safe-change window. The sentence about partners fixing vulnerabilities is a statement of activity, not a result. A separate programme the announcement mentions, begun in June for US state, local, tribal and territorial governments, is not this one and is not assessed here. For how Anthropic's access tiers work, see our 6 October briefing on the Cyber Verification Program.

For UK organisations that consume open source: expect more upstream fixes, and read severity from the maintainer

If maintainers act on reports faster, the first thing a consumer sees is a release, not a report. Anthropic's agreement treats each report as its "confidential material", so you will not read the scanner's severity or its reproducer. You will read whatever the maintainer chooses to publish, and your clocks run from that release.

UK guidance read on 9 October 2026, and what it means here. The last column is our reading, not the publisher's

UK source	What it says	What it means here
NCSC, update by default (version 2.1, reviewed 1 May 2026)	Internet-facing services: 5 days. Operating system and applications: 7 days, applied automatically as published. Internal and air-gapped: 14 days. For all updates, whatever the severity.	The clock starts at release. A wrong severity in a report does not change it.
Cyber Essentials requirements v3.3, April 2026	Updates for vulnerabilities the vendor calls critical or high risk, scoring CVSS v3 7 or above, or with no severity given: within 14 days of release.	A release note that repeats an inflated rating can start this clock. Check the advisory, not the headline.
NCSC, "Preparing for a vulnerability patch wave", 1 May 2026	Plan to deploy updates "quickly, more frequently, and at scale", put the external attack surface first, and seek assurance from commercial and open source supply chains.	OSS Scanner is a named new source of upstream volume.
NCSC, "10 questions to ask when using AI models to find vulnerabilities", 11 May 2026	Have "a process to manage any vulnerabilities that AI finds", and avoid spending everything on "finding vulnerabilities" with "nothing left to fix them".	Written for those running the models. It reads as well for those receiving the output.
Software Security Code of Practice, GOV.UK, updated 15 January 2026	Principle 3: publish a vulnerability disclosure process (3.2) and manage vulnerabilities in software components (3.3). An open source maintainer "bears no formal commitment to their onwards supply chain"; the risks "must be managed by end-users or proprietary developers using open-source code in their software".	The code puts open source risk on the consumer. OSS Scanner puts the first human review on the maintainer. No contract links the consumer to the maintainer.

Whether volunteers can match a funded team is the open question. Our 6 October briefing on a browser release found fourteen credit lines naming an AI tool, all reported 6 to 12 days before release. That was one vendor's release, with fixes that followed reports its team could read. Here the reports are unreviewed and the receivers are often, in Anthropic's words, "small teams of volunteers".

For UK operators of essential services: the CAF asks for your evidence, and a supplier's programme is not an indicator

The NCSC's Cyber Assessment Framework (version 4.0, reviewed 6 August 2025) is written for operators of essential services in energy, healthcare, transport, digital infrastructure and government. Principle B4, system security, includes B4.d, vulnerability management. At the achieved level it expects announced vulnerabilities for all software packages and systems supporting the essential function to be "tracked, prioritised and mitigated (e.g. by patching) promptly", with regular testing verified by third parties. Its guidance tells operators to "carefully consider their approach to the testing of live Operational Technology". Principle A4, supply chain, carries an indicator that reads: "If open-source software is used, you have taken appropriate and proportionate steps to establish and maintain sufficient confidence in its security for its use."

The NCSC's patch wave advice says update by default "may not apply in some circumstances (such as for safety-critical systems or operational technology)". The Five Eyes leaders' statement of 22 June 2026 says delays in patching "increase risk, especially for operational systems with long update cycles".

Nothing in B4.d or A4.b gives credit for a supplier's membership of a programme. The evidence is the operator's own, on its own estate. If an OT security provider is one of the eleven, the useful questions are the ones the announcement leaves unanswered: which model, what data leaves the site, who the on-site engineers are, and who signs off a change to a running system.

For a UK maintainer or open source programme office: seven decisions before the pull request

Anthropic says there is "little downside to signing up if you're eligible". The agreement and the repository show where a downside would sit. A company that employs a maintainer, or runs a programme office across several projects, should settle these first.

A company that sells software built on open source has a second duty. The Software Security Code of Practice asks vendors to publish a vulnerability disclosure process and to manage vulnerabilities in the components they ship. A scanner report that reaches a maintainer in your organisation is also an input to that process, and it arrives with a reproducer. Treat it as sensitive: restrict who can read it, and do not forward it beyond the authorised maintainers the agreement allows.

What to do, in the order worth doing

The question this launch leaves

Anthropic's outside reviewers had examined 6,123 of 29,439 findings, and Anthropic calls that human step the rate limiting one. OSS Scanner moves the step to a maintainer, with an expected accuracy whose definition the pages do not give. Who in your organisation, or in the supply chain you depend on, has agreed to be that reviewer, and what happens to your patch clocks when they cannot keep up?

Key facts