A UK government study ran 192 models through 8 safety benchmarks. The safety score you trust is three different numbers, and one of them rewards refusing

What the paper actually did

On 5 August 2026 four researchers, two of them at the UK AI Security Institute, published Item Response Theory for AI Safety (arXiv:2608.05086). They took eight established AI safety benchmarks, ran 192 language models through them, and asked a question almost nobody asks of a leaderboard: does the number measure one thing, and does it measure the thing it claims to?

It is, by their own account, the largest psychometric analysis of language-model safety evaluations done to date. The tool is Item Response Theory, the same statistical machinery used to work out whether an exam question is actually measuring ability or just measuring who has seen it before. Point that at safety benchmarks and three findings fall out, and the industry has been quoting the first one wrong.

Finding one: "safety" is three different things

A single benchmark prints a single number, and everyone treats that number as how safe the model is. The paper finds the number is a blend of three factors that do not move together: refusal strictness, truthfulness, and contextual harm. Most of the variation between models across all eight benchmarks is explained by these three, and they are genuinely distinct axes.

A single safety score, split into the three factors it contains

One “safety” number, three answers that disagree

Aggregate
78

Refusal strictness
86

Truthfulness
61

Contextual harm
44

• Refusal strictness. How readily it declines. High here is what games the score.
• Truthfulness. Whether what it does answer is correct.
• Contextual harm. Whether it avoids harm in context, not by blanket refusal.

That matters because a respectable aggregate can hide a weak score on the axis you care about. A model that refuses readily and is scrupulously truthful can still be poor at avoiding harm in context, and its headline safety score will not tell you. You bought a number that averaged away the answer.

Finding two, the one that should worry a buyer: the score rewards refusing

Here is the specific mechanism, and it is the part the coverage led with because it is the most damning. Two of the eight benchmarks pull in opposite directions. Push a model to refuse more, and its score on a harm benchmark improves while its score on an over-refusal benchmark collapses. Because the aggregate weights them a certain way, a model can raise its overall safety rating simply by declining more requests, even as it becomes less useful.

Turn up refusal, and the safety score follows it up while usefulness falls

Turn up refusal, watch the safety score follow

Refusal rate: the model just says no more often

Safety score

95/100

Goes up. This is what the leaderboard prints.

Actual usefulness

18/100

Goes down. This is not on the leaderboard.

Every practitioner has met the model that answers "I can't help with that" to something obviously fine. This paper is the measurement showing that behaviour is not a bug the labs are failing to fix. It is a behaviour the benchmarks actively reward. The blanket refusal that annoys your users is a rational move for climbing the leaderboard.

Finding three: ten questions can replace a whole benchmark, and can catch a swapped model

The last result is quieter and has the longest reach. Because IRT works out which test items are actually informative, the authors could recover a model's full benchmark score from a handful of well-chosen questions. Roughly ten adaptively selected items reproduced several benchmarks with lower error than a random sample of the same size, cutting evaluation cost by 97 to 99%.

192

Models analysed

The largest psychometric study of LLM safety evaluations to date.

8

Safety benchmarks

Decomposed into three underlying factors, not one.

3

Factors that matter

Refusal strictness, truthfulness, contextual harm. They disagree.

97-99%

Cheaper to evaluate

About ten adaptively chosen items recover a full benchmark score.

The same machinery does something a security team should sit up for. Because a model has a stable psychometric fingerprint across these items, IRT can detect naive sandbagging, a model behaving more cautiously when it senses it is being tested, and it can detect when the model behind an API has quietly changed. If you pay for model X on an endpoint and the provider swaps in a cheaper or differently tuned model Y, a ten-item probe can catch it. That is a continuous-assurance capability, not just a research result.

Why this is a governance problem, not an academic one

Point the three findings at the documents that increasingly cite benchmark scores as evidence.

Where a single safety score gets used, and what this paper does to it

A safety score is used to...	...and the decomposition means
Rank models on a leaderboard	The ranking depends on how three unlike factors were weighted, a choice the leaderboard rarely shows.
Satisfy a procurement or assurance checklist	A passing aggregate can hide a failing factor. Ask which factor, or you have not asked anything.
Evidence a claim under a framework or the EU AI Act	An auditor citing one number is citing an average of three, at least one of which the deployment may depend on.
Show a model got safer between versions	A higher score can mean more refusal, not less harm. The improvement may be the model getting less useful.

This is the same defect this site keeps finding in published artefacts, in a new place. A status page named components and never a cause. A breach notice left the data types blank. Here a safety score names a virtue and averages away whether the model has it. In every case the artefact is accurate and answers a narrower question than the reader is asking.

It also lands squarely on the cyber threat intelligence guide published here last week, which opened with the Butler Review's lesson that a confident number with its uncertainty stripped is more dangerous than no number at all. A single safety score is exactly that: a confident figure that has thrown away the disagreement inside it.

What to do with this

The position

The benchmarks are not fraudulent and the labs running them are not cheating. The paper is careful and constructive: its recommendation is that frontier labs and evaluators adopt IRT, because it reads, shrinks and audits these tests better than the aggregate does.

The uncomfortable part is what it implies about the last two years of safety reporting. A great deal of it rests on single numbers that, measured properly, turn out to be three numbers in a trench coat, with at least one of them, over-refusal, actively gamed by the way the score is built. When a government safety institute publishes the measurement showing your headline metric rewards the wrong behaviour, the correct response is not to defend the metric. It is to report the three factors and let people see which one they are relying on.