Anthropic says 1 in 47,000 agent decisions was blocked. The number it does not publish is the catch rate

Three numbers published, and the fourth one named but not given

Anthropic has published a set of measurements about its own use of AI, including, for the first time, figures on how it oversees the agents running inside the company. The oversight section is short and specific. As of August 2026 there were approximately 30,000 agents doing research and engineering work at any one time on its most used internal platform. Every action those agents take passes an online monitor before it executes, usually within seconds. Over a billion decisions were analysed across August, and 0.002 per cent of them, which the post puts as about 1 in 47,000, were blocked.

A second, offline monitor reads everything after the fact. It flags roughly one to two transcripts in every thousand, which comes to about 100,000 transcripts a week. Classifiers resolve most of those, and approximately 50 a week reach a human.

The three metrics Anthropic proposes as an industry standard are coverage, review latency and escalation rate. Each of them describes the behaviour of the monitor. None of them describes the behaviour the monitor missed.

What these figures do not establish. A blocking rate is not an error rate: Anthropic says so, and it is the first thing to hold on to. It is not a misbehaviour rate either, because it counts what one system decided to stop, not what the agents tried to do. It does not establish that agents are safe, or unsafe. It does not cover Anthropic's other platforms, because the post says plainly that these measurements cover one platform only. And it establishes nothing about any other company, because nobody else publishes anything comparable.

What it does establish is a baseline that did not exist a fortnight ago, and a vocabulary. That is worth more than the numbers themselves.

What was actually measured

The system has two halves, and they are built for different failure modes. Online monitors are for harms that happen fast and cannot be undone, and the post gives an example: "an agent copying model weights out of our systems". Offline monitors are for harms that develop slowly and can be reversed, such as signs of model misalignment.

The oversight figures for August 2026, as published

Measure	Online monitor	Offline monitor
Coverage	100 per cent of actions, before execution	100 per cent of actions, after the fact
Latency	Usually within seconds	Flags reviewed by classifiers, then humans
Human review	Blocked actions reviewed within one week	Approximately 50 escalations a week
Rate	0.002 per cent blocked, about 1 in 47,000	Roughly 1 to 2 transcripts flagged per 1,000
Basis	Over a billion decisions across August	Roughly 100,000 flagged transcripts a week

Two design choices underneath are more interesting than the numbers. Each agent has an individual identity that is tied to everything it produces and that persists across model upgrades, so an agent's record is continuous even when the model powering it changes. And agents communicate through a shared, open messaging system rather than privately, with every message tied to the posting agent and cross referenced with transcripts.

The stated reason for identity is oversight: every action can be audited per agent over time. The stated reason for open messaging is that agents can see and correct each other, and that monitoring can traverse the conversation between them. Both are architectural decisions taken to make monitoring possible at all, which is the part most organisations deploying agents have not done.

The arithmetic underneath the percentages

Percentages at this scale hide their own size. Working the figures back into counts changes how they read.

Over a billion decisions at 0.002 per cent gives at least 20,000 blocked actions in August. Anthropic says humans review any blocked action within one week, so that is on the order of 4,600 human reviews a week, every week, just for the online half. That is not a rare event handled by an on call engineer. It is a staffed function.

On the offline side the funnel is far steeper. Roughly 100,000 transcripts are flagged a week and approximately 50 reach a human, which is about one in every 2,000 flags, or 0.05 per cent. Everything else is resolved by classifiers. The quality of that reduction is the quality of the whole offline system, and it is not reported.

Counts derived from Anthropic's published rates, computed for this briefing

Derived figure	Working	Result
Blocked actions in August	over 1,000,000,000 multiplied by 0.00002	at least 20,000
Blocked actions a week	20,000 divided by 4.43 weeks in August	about 4,500
Share of flags reaching a human	50 divided by 100,000	0.05 per cent, or 1 in 2,000
Transcripts produced a week, inferred	100,000 divided by a flag rate of 1 to 2 per 1,000	50 million to 100 million

The metric Anthropic names and does not report

Coverage, latency and escalation rate all answer the same shape of question: what did the monitor do? The question they cannot answer is what the monitor failed to do. In detection terms, the published metrics are about throughput, and the missing one is recall.

Anthropic is not hiding this. In the same post it writes that developers "should share how often known agent misbehavior is caught by monitors, and should have third parties do this kind of testing independently". That sentence describes a catch rate. It appears as a recommendation to the industry rather than as a number about Anthropic.

The post also records the honest limit in its own appendix: "At this scale of agent activity, it is hard to be certain we have a complete picture of all possible kinds of behaviors, or that we are making these patterns adequately legible to human oversight. We have been doing this kind of monitoring for months, not years."

What the three published metrics answer, and what they do not

Question	Answered?
What share of agent actions is monitored?	Yes. 100 per cent, on this platform
How quickly is an action reviewed?	Yes. Seconds for the online monitor
What share is blocked or flagged?	Yes. 0.002 per cent blocked, 1 to 2 per 1,000 flagged
What share of real misbehaviour is caught?	No. Not published
What share of blocks were false positives?	No. Not published
How accurate are the classifiers that resolve 99.95 per cent of flags?	No. Not published

There is one external check on the record. The post says METR has independently red teamed Anthropic's offline monitoring platform in the past, and that the company is now setting up external third party evaluators. A red team exercise is the closest thing to a catch rate that exists here, and its results are not in this post.

This is the single most useful thing for a reader to take away, because it generalises. Any vendor who sells you agent monitoring can tell you their coverage and their alert volume tomorrow. Ask what fraction of planted misbehaviour their monitors catch, and who planted it.

How the 26 per cent was produced

The headline that travelled furthest from this post was a different number: Claude now leads 26 per cent of Anthropic's AI research and development work, with more than 90 per cent at or above the level the scale calls "AI collaborates", and no measured subset running fully autonomously. The scale itself is Epoch AI's, running from no AI involvement to fully autonomous.

The method matters more than the figure. For each week of July 2026, Anthropic sampled 20 per cent of staff in the departments that make up the model research loop. A Claude research agent read each sampled person's week through Slack and internal documentation and listed their tasks, producing about 15,000 granular tasks. Claude then organised those into a tree of 542 nodes, 378 of them leaves, which is frozen so later measurements run against the same basket. For each node, a Claude agent researched how that work is done and an independent Claude judge assigned an automation level. Weights come from person time: each person contributes one unit a week, split evenly across the tasks they worked on.

So the index is Claude measuring Claude's contribution, judged by Claude, weighted by a proxy for importance that Anthropic calls a crude approximation, against a basket frozen in July 2026. Anthropic says all of this in the appendix, tested the frozen basket problem by rebuilding it from January 2026 data, and found no rise in novel tasks. That is a serious attempt to break its own number.

It is still a self report. The company chose the scale, built the pipeline, ran the judge and published the result, and no third party has re-run it.

The compute figure and its conservatism

For one week, 13 to 20 July 2026, Anthropic sorted its compute into categories and asked how much of what went to AI research and development went to safety. The answer was about 6 per cent, and about 12 per cent of the compute that went to AI driven AI research and development.

The classification was done by a prompted Claude classifier working from run metadata and code, with the prompt itself partly reproduced in the appendix. Rather than classify all of the week's nearly 10,000 runs, Anthropic sampled about 14 per cent, weighted towards the runs that used the most compute. Where an agent's session transcript was inaccessible because the work was compartmentalised, the run was classified by the user's team or defaulted to AI research and development.

The post calls these estimates deliberately conservative, and the reasoning is sound: work that advanced capability as much as safety was counted as capability, and safeguards classifiers were excluded. It also says the definition "is one reasonable choice among many; a different developer, or a regulator, might draw the line differently", and that classifier agreement with human reviewers ran within one or two percentage points.

A six per cent figure with a two point classifier disagreement band is a number to treat as a range, not a point.

What to do with this

Most readers do not run 30,000 agents. The useful part is the set of questions this post makes it possible to ask of anyone who does, including your own vendors and your own platform team.

The question this leaves

It would be easy, and wrong, to read this as a company marking its own homework and finding it excellent. The post is unusually candid: it names the judge model problem, publishes the agreement statistics that undercut its own precision, admits the compute definition is one choice among many, says the monitoring has run for months rather than years, and recommends that third parties check all of it. Nobody else has published anything like it, which is the only reason we can have this conversation at all.

But every published number describes the monitor, and the quantity that would tell you whether the monitoring works is the one that requires somebody outside to plant something and count what got through. Anthropic says that test should happen and that METR has done a version of it before. The result is not in the post.

So when the next frontier lab publishes its oversight metrics, and they will, the question to ask is not how many actions were blocked. It is this: of the things you deliberately tried to slip past your own monitors, how many got through, and who was holding the stopwatch?