Anthropic's 0.002% election-policy violation rate is about 5 in 250,000 conversations, with no count of actions

A rate of 0.002% is about five conversations in 250,000

Anthropic's post of 14 October 2026, "Preparing for the 2026 US midterms", says that requests which violated its election integrity policies appeared in "about 0.002% of conversations". In a sample of 250,000 that is about 5 conversations (derived: 0.002% of 250,000). The post gives percentages, not counts. Converted the same way (derived), 5.75% is about 14,375 conversations, 0.51% is about 1,275, and the figure for asking whom or what to vote for, which the post's own chart labels "at most about 0.01%", is about 25 or fewer.

A rate that small tells a reader how seldom Anthropic's review found a violation in the analysis it ran. It does not establish how accurate that method is, how many of the five a person confirmed, what happened to them, whether the models Anthropic now cites were the ones answering, or what happens outside the US consumer app. It is also not a precise number. An exact 95% interval for a count of about 5 runs from roughly 1.6 to 11.7 conversations (derived, Poisson), and the rounding of "about 0.002%" alone spans 3.75 to 6.25.

This briefing read the whole post, both footnotes and its three charts, then every page it links: the April election safeguards update, the 2024 elections post, the political evenhandedness write-up, the Anthropic Insights page and its paper, the Usage Policy (current and previous versions) and the policy update, the campaigns guide, the threat intelligence report, and the Democracy Works and evaluator pages. It then compared the post's "99%" with the three system cards that report it. Everything was read from 08:46 BST on Thursday 15 October 2026. No independent coverage of the 14 October post was found in the searches run. The UK section reads the UK pages that apply here, because the UK has no midterms.

Interest, in one sentence: Anthropic sells Claude and wrote the evaluation that its models score 99% on, so every figure below is the vendor's own and is attributed as such.

Four figures, and how the post says each was made

The four published figures, the counts they imply in a sample of 250,000 (derived), and what the post and its footnotes say about each. Source: the 14 October 2026 post and its charts.

Published figure	Count in 250,000 (derived)	What the post says about how it was made
5.75% touch politics, public policy or social issues	About 14,375	US Claude.ai consumer conversations, 21 to 27 September 2026, "classified" by Claude using a privacy-preserving method.
0.51% are about elections or voting	About 1,275	Same sample and method.
"About 0.01%" ask whom or what to vote for. The chart says "at most about 0.01%"	About 25 or fewer	The chart shows "the upper end of a 95% confidence interval". The point estimate and the interval are not printed.
"About 0.002%" violated election integrity policies	About 5	Footnote 2: "a separate analysis of 250,000 Claude.ai conversations, which were reviewed against our election integrity policy". Who reviewed, which week and whether it was US only are not stated.

The 0.01% is a ceiling, not an estimate. The text says "about 0.01%". The chart beside it says "at most about 0.01%" and explains that, because such requests are so rare, it shows the top of a 95% interval. If that interval is an exact one, a top of about 25 fits an observed count of roughly 14 to 18 conversations, which would be a rate of about 0.006% to 0.007% (derived inference from the caption; the post prints neither number).

The 0.002% comes from a different analysis. Footnote 2 calls it "a separate analysis of 250,000 Claude.ai conversations". It does not say the conversations are the same 250,000, are American, are consumer conversations, or fall in the same week. The figure should not be read as a share of the sample behind the other three. The earlier count on record is from December 2024, when Anthropic reported "approximately 100" election-related enforcement actions worldwide over the year, including warnings and some account bans. The 14 October post gives no count of actions at all.

Claude classified the sample, and the post states no human review count. The chart caption says Claude classified the conversations. Anthropic's own page on the method, then called Clio and now Anthropic Insights, says its steps are "powered entirely by Claude, not by human analysts", and that only higher-level clusters are visible to human analysts (a 2024 description). So, on that description, no person reads the sampled conversations for the topic shares, and the post gives no number. For the violation figure, the reviewer (a model, a classifier or a person) is not stated. The post separately says its threat intelligence and enforcement teams "review flagged activity and take action when they confirm a violation, including banning accounts", with no number reviewed, confirmed or acted on.

Sampling error is not classification error. With 14,375 or 1,275 conversations the sampling margins are small (derived, Wilson 95%: 5.66% to 5.84%, and 0.48% to 0.54%). That says nothing about whether the labels are right. The 2024 paper behind the method reports 94% accuracy on a synthetic test set and 84% on a synthetic set of concerning conversations, using the Claude 3 models of the time, and describes its outputs as "preliminary" and needing "additional validation" before being used for decisions. The 2026 post does not say which model classified the sample or give any accuracy figure for it.

The sample week, and the models behind the 99%

The post names Claude Opus 5.5, Sonnet 5.5 and Haiku 5.5 as "our most evenhanded to date, scoring 99%". Anthropic's launch pages carry the dates below. The sample week was 21 to 27 September 2026.

Launch dates from Anthropic's launch pages, set against the seven-day sample week of 21 to 27 September 2026 (days counted are derived).

Model named in the 99% claim	Launch page date	Days of the sample week it existed (derived)
Claude Opus 5.5	22 September 2026	6 of 7
Claude Sonnet 5.5	28 September 2026	0: launched 1 day after the week ended
Claude Haiku 5.5	7 October 2026	0: launched 10 days after the week ended

The post does not say which models answered the 250,000 sampled conversations. Whatever they were, two of the three models in the 99% claim did not exist yet, and the third existed for six of the seven days. That is not an error, because the two statements answer different questions: what people asked in September, and how newer models score on a test. It does mean the rates in the first half of the post cannot be read as measuring the behaviour of the models named in the second half.

Timing relative to the vote (derived): the last sample day is 37 days before Election Day on 3 November 2026, the post is 20 days before it, and the post expects election-related share to rise. In 2024 Anthropic reported election-related activity rising from under 0.5% to just over 1% of use in the weeks before the US election, across Claude.ai and the API, on different definitions. That is context, not a forecast.

The 99%: one setting, and the number beside it

The post links its "political evenhandedness evaluation" to a November 2025 write-up. That write-up describes an automated test of 1,350 prompt pairs across 150 topics, in which a Claude model grades whether a model treats opposing requests with comparable depth, and Anthropic open-sourced the dataset. It says the test is single-turn and "primarily" about current US political discourse, and that "there is no agreed-upon definition of political bias, and no consensus on how to measure it". A footnote adds that "API users aren't required to follow these standards" and can configure Claude to reflect their own views within the Usage Policy.

The three system cards report the same evaluation in two settings: claude.ai with its default system prompt, and the API without one. They also report a multi-turn election integrity test. The post leads with the first and does not mention the others.

Political even-handedness (1,350 prompt pairs) and multi-turn election integrity, from Anthropic's system cards for each model. Higher is better in every column. Margins in brackets are the cards' own.

Model (launch)	Even-handedness, claude.ai with system prompt	Even-handedness, API, no system prompt	Multi-turn election integrity, claude.ai
Opus 5.5 (22 Sep)	99.5%	99.4%	85% (plus or minus 6)
Sonnet 5.5 (28 Sep)	99.0%	97.9%	86% (plus or minus 6)
Haiku 5.5 (7 Oct)	99.1%	97.7%	98% (plus or minus 3)

Read plainly (derived reading): the post's "99%" is the claude.ai column, 99.5, 99.0 and 99.1. On the API the same cards give 99.4, 97.9 and 97.7. One percentage point of 1,350 pairs is about 13.5 pairs, so Sonnet 5.5 on the API left about 28 pairs short of even-handed and Haiku 5.5 about 31 (derived). The multi-turn column is the related measure for the weakness the post itself describes. The post says outside analysts at two institutes found Claude "reliably refused a single harmful request, but could miss a malicious campaign when it was spread across many individual requests that looked harmless on their own". The cards put Opus 5.5 at 85% and Sonnet 5.5 at 86% on that kind of test on claude.ai, so about 15% and 14% were not handled appropriately (derived). The cards call the falls from earlier models within the margin of error. Haiku 5.5 scores 98%. The post prints none of these figures. The size of the multi-turn test set is not given in the three cards read.

What lifted the score. The Opus 5.5 card says most of the pairs it handled better than Opus 5 were requests for one-sided content such as persuasive essays, where it more often gave both sides "the strongest version of the requested view along with its main counterarguments". The post says two of its outside reviewers found Claude "sometimes added unsolicited caveats and counterarguments, such as inserting an opposing view into an advocacy email". These look like the same behaviour seen from two sides (inference): the evaluation rewards adding the counterargument, and the reviewers count it as a shortfall. The post does not say how the two were reconciled. The card also reports that Opus 5.5 acknowledged opposing perspectives less often on the API than Opus 5 (26.9% against 46.9%), which shows how much the score depends on how the test is defined.

One more line from the Opus 5.5 card matters to anyone defending election systems. Anthropic's policy experts found that the model was "sometimes more willing" than Opus 5 to accept a user's claimed authority in a scenario involving election infrastructure, a finding the card reports without a rate. A model does not verify authority. The post says Anthropic has "strengthened our cyber protections for election infrastructure and campaigns" without a measure.

Three independent groups: what each tested, and when

The post says "we asked three independent groups to review how Claude handles free expression". It gives no dates and names a model version for only one group. The table sets what the post says against what could be found.

The three groups as the post describes them, the model version named, and what a search of each group's own site and the web found on 15 October 2026.

Group	What the post says it did	Model version, and a public report?
The Future of Free Speech (think tank at Vanderbilt University)	Tested Claude on "hundreds of requests on contentious topics". With CIP, found Claude rarely refused but "sometimes added unsolicited caveats and counterarguments".	Version: not named. Public: no report matching the review was found. Its October 2025 report tested "Anthropic Claude Sonnet 4" among eight chatbots on 500+ prompts, which may or may not be the same work.
Foundation for American Innovation	Compared matched left and right prompts. Found Claude Sonnet 4.5 "didn't always hedge evenly" and was more likely to soften arguments for right-leaning positions.	Version: Sonnet 4.5. Public: no report found on its site or by search.
Collective Intelligence Project	Surveyed "more than 2,000 people" in the US, UK and India to build an evaluation, with shared principles tested across groups.	Version: not named. Public: none found. No per-country count and no UK result.

Versions against the claim (derived): the only named version, Sonnet 4.5, is at least three releases before Sonnet 5.5 on the Sonnet line as Anthropic's own documents name it (4.5, 4.6, 5, 5.5). In April 2026 Anthropic described the three groups as "currently working with" it on "a broader review", when the models its post tested were Opus 4.7 and Sonnet 4.6. None of the three groups is stated to have tested Opus 5.5, Sonnet 5.5 or Haiku 5.5, and Haiku 5.5 launched a week before the post. The post says Anthropic has since improved how Claude answers yes-or-no questions and when it adds caveats. It does not say anyone re-tested.

"Independent" here means the groups are not part of Anthropic. It does not tell a reader whether Anthropic funded them, set the scope, saw drafts or could stop publication, and the post does not say. The Future of Free Speech's report pages name other sponsors for its 2025 and 2026 reports. The Collective Intelligence Project's home page says only that it is "Funded by leaders in AI". The post also says the Institute for Strategic Dialogue and the McCain Institute pressure-tested the safeguards in the US, Israel and Brazil. Neither organisation's home page carries a report on it, and no public report was found.

Stated and not stated: the record, line by line

"Not stated" means not found in the post, its footnotes or the pages it links, all read. It does not mean the information does not exist.

What Anthropic's 14 October 2026 post and the pages it links state, and what they do not.

The question	Stated	Not stated
How accurate is the classifier?	Claude classified the sample with a privacy-preserving method.	Accuracy, false positives and negatives for 2026, which model, any human check of a sample.
What is the 0.002%?	A violation share from "a separate analysis" of 250,000, reviewed against the policy.	The count (about 5, derived), the reviewer, the week, the country, how many were confirmed.
What was done about them?	Teams review flagged activity, act on confirmed violations, ban accounts. Two cases described.	Numbers flagged, confirmed, banned, warned. Time to detect. Whether any of the five led to action.
What is the 99%?	Opus, Sonnet and Haiku 5.5 score 99% on Anthropic's evaluation.	That it is the claude.ai figure, that the grader is a Claude model, and the 85% and 86% multi-turn figures.
Were the reviews current?	Three groups reviewed Claude. One found Sonnet 4.5 hedged unevenly.	Dates, versions for two groups, publication, terms, funding, any re-test of the 5.5 models.
Outside the US consumer app?	Tests in the US, Israel and Brazil. Work "extends to elections around the world".	API and third-party products, Claude Code, enterprise, other countries, the UK.
What happens off Claude?	Because early stages happen on Claude, "we often have a chance to stop these efforts before they ever reach the public".	The share of misuse that happens on other models or elsewhere. The audience any case reached.
The two midterm cases?	A fake news site used Claude for US midterm coverage favouring Russian, Iranian and North Korean narratives. Others made fabricated news for advertising revenue. Accounts banned.	Dates, account numbers, how found, audience. The linked September report, covering December 2025 to August 2026, does not contain them.
Official voting information?	Democracy Works information in Claude.ai for US users "later this month". Election-night results "directly in their conversation".	A date, the source of the results, state and local coverage, how errors are corrected.
Which policy applies?	The post links the "election integrity policies" and says this year's update was announced last week.	That the linked version takes effect on 12 November 2026, nine days after Election Day (derived). Until then the previous version, effective 15 September 2025, would apply (inference).

The policy row needs the detail. The Usage Policy page the post links carries the header "Effective November 12, 2026" and a link to a previous version, effective 15 September 2025. The 8 October update post says the new text takes effect on 12 November and removes the "blanket prohibition on personalized vote and campaign targeting", keeping the behaviours that motivated it under other sections. The 0.002% review was against the policy in force in September, which still carried the ban on personalised vote and campaign targeting that the update removes. The post does not say which version it reviewed against. Anthropic's campaigns guide, dated October 2026, adds that election laws "may require public communications to contain certain disclaimers" and that users should ensure Claude-assisted content carries them for their jurisdiction.

Six friendly names, and what each hides

A label is not a measurement. Six appear in the post, and each stands in for a control or a figure.

Labels in the post, and what the record shows behind each.

The label	What the record shows	What it hides
"Evenhanded"	A score on an automated test of 1,350 prompt pairs, written by Anthropic, graded by a Claude model.	The claude.ai setting, a lower 97.7 to 97.9% on the API for two models, and gains the Opus 5.5 card ties mostly to adding counterarguments.
"Safeguards"	Classifiers, human review of flagged activity, a policy, a banner and a data feed.	No detection or action counts, and an 85% to 86% multi-turn figure for two models on its own card.
"Election integrity"	The name of a policy and a 300 plus 300 prompt test.	The policy in force in September was the 2025 text, and the page the post links is not in force until 12 November.
"Privacy-preserving"	A way to count topics without people reading the chats.	It says nothing about whether the labels are right. The method page says the steps are run by Claude, and the 2024 paper calls outputs preliminary.
"Independent"	Groups outside Anthropic that it asked to review Claude.	Dates, funding, terms, a public report, and any test of the models now being named.
"Official voting information"	A nonprofit's compilation, drawn from election offices and some third-party tools.	It is not issued by election authorities, and it was not live when the post was written.

The last row rests on Democracy Works' own page for its data service. It says the data is gathered "from official government sources", with reliance also on third-party tools including the Google Civic Information API, LegiScan and Ballotpedia, and "two rounds of quality assurance". The post itself says the responses draw on information "that Democracy Works collects from state and local election sources". Democracy Works describes itself as a nonpartisan nonprofit. That is a compilation by a third party, and the label "official" should be read that way.

Method is separate from accusation. The post does publish its sample method, two misuse cases, the dataset behind its evaluation and the shortfalls its reviewers found (uneven hedging, multi-request campaigns), and nothing here says the figures are wrong. The gap is in what is reported next to them. This site has covered another vendor's influence operations report, which also gave no account counts or confidence.

The UK: no midterms, and where the rules that apply here stop

Nothing in the post is about a UK election. Anthropic's 2024 election post listed the UK Electoral Commission among the sources it pointed users to for UK elections. The 2026 post names only TurboVote and Democracy Works, for the US. The Collective Intelligence Project surveyed people in the UK, and no UK result was found. A UK reader has three questions: which UK rules reach AI in campaigning, which reach a chatbot, and where those stop. The table reads the pages as published on 15 October 2026.

UK instruments as read on 15 October 2026 on the regulators' own sites and legislation.gov.uk, and where each stops.

UK instrument	What it says	Where it stops
Electoral Commission, campaign material pages (June and July 2024) and principles for campaigning	"We do not regulate the content of campaign material." It expects AI-generated campaign material not to mislead and to be labelled clearly. Its six principles are advisory and say parties "should not use or amplify" false or misleading information, including deepfake material. It says it will correct false information about voting processes.	An expectation and good practice, not law. Written for the 2024 general election. Nothing on a chatbot answering voters.
Elections Act 2022, Part 6, and the Commission's statutory guidance on digital imprints	Digital campaign material from parties, candidates and some campaigners needs an imprint, including text, audio and visual content, and paid adverts. The Commission enforces for parties and campaigners, the police for candidates.	The guidance read has no mention of chatbots, artificial intelligence or generative AI. Whether a personalised chatbot reply is imprint material is not answered. Telephone calls, SMS and party emails to members are excluded.
Representation of the People Bill (introduced 12 February 2026)	Extends imprints to third-party campaigners that are not recognised, and makes the Commission the main enforcer. Passed the Commons on 2 September (411 to 102), Lords second reading 14 September, Lords committee listed from 2 November, provisional.	A bill, not law. The Commons Library records that the government rejected amendments on deepfakes, AI-generated content and misinformation as "better regulated by Ofcom under Online Safety Act provisions".
Online Safety Act 2023, Ofcom's 3 February 2026 update, and the Crime and Policing Act 2026, section 248	Ofcom says a chatbot is outside the Act if it only talks to the user, does not search several sites and cannot make pornography. Section 248 (in force at Royal Assent, 29 April 2026) lets the Secretary of State make regulations on "illegal AI-generated content" and priority offences.	A power, not a duty: no regulations were found. It concerns illegal content, not lawful but one-sided answers about voting.
NCSC Defending democracy (published December 2023, reviewed May 2024)	Cyber advice for political organisations and election coordinators: multi-factor authentication, anti-spoofing, phishing, backups, the Early Warning service. It notes generative AI makes spear-phishing more convincing.	Cyber security only. Nothing on staff using chatbots or on AI-made disinformation.
ICO guidance on political campaigning	UK GDPR, the Data Protection Act 2018 and PECR apply to campaigners. Profiling and micro-targeting call for a data protection impact assessment. Psychometric targeting to provoke a reaction is "likely to be unfair".	Marked "under review" after the Data (Use and Access) Act. No mention of AI chatbots in the pages read.

Put together (this briefing's reading of the pages, not a legal view): the Electoral Commission does not regulate content, the Government, at committee stage, pointed deepfake and AI amendments to Ofcom, and Ofcom says a one-to-one chatbot is outside the Act unless it searches several sites or shares output. The power in section 248 is the route to change that, and it is limited to illegal content. None of the instruments read puts a duty on the provider of a one-to-one chatbot to give accurate or balanced answers about voting. The duty a UK campaigner, council or company has today comes from its own obligations: imprints, data protection, the Commission's expectations, and the provider's contract.

One more point for a UK team. Anthropic's guide pushes disclaimer duties to the user, and its Usage Policy makes users responsible for the agents they deploy. A UK team that uses a vendor's model for voter outreach is the party that must meet UK law, whatever the vendor's policy says.

What to do, in the order worth doing

Three audiences, each in order. Every item rests on a control one of the sources above names or on a figure this briefing derived. None needs access to the vendor's data.

What could not be verified

• The classifier's accuracy in 2026. The post gives none. The 2024 figures belong to a different version of the method.
• The 0.002% in detail. The reviewer, the week, the country and the count are not stated. The count of about 5 is derived on the assumption that 250,000 is the base.
• Which models served the sample. Not stated.
• The three independent evaluations. No public report matching the review as described was found for any of the three groups. The Future of Free Speech's October 2025 report was read only as the closest public work. Dates are not stated.
• The pressure test. No public report from the Institute for Strategic Dialogue or the McCain Institute was found.
• The two US midterm cases. They rest on the post alone. They are not in the September threat report as read.
• Democracy Works inside Claude. Not live when read. The feed's coverage and the source of election-night results are not stated.
• Regulations under section 248. None was found as of reading. That is a search, not proof that none exists.
• Coverage. No independent coverage of the 14 October post was found in the searches run. One news article on the April update, found by search, returned HTTP 403 and was not used.
• Time-stamped. Sources were read between 08:46 and 09:10 BST on 15 October 2026. The post's own page was modified at 21:01 UTC on 14 October. A bill's stage, a page's wording or a model's score could change within hours.

The question that exposes the gap

Of the roughly five conversations in 250,000 that Anthropic says broke its election rules, how many did a person confirm, how many led to an action, how long did that take, and how would anyone outside Anthropic find out?

Key facts