Home Office asylum summaries: 45 of 203 failed first review, and omissions are not a stated failure test

45 of 203 failed first review, and the checks cover about 1 per cent of the output or less

The Home Office told the Open Rights Group (ORG), in a freedom of information response of 11 September 2026, that subject matter experts had made 259 checks of summaries from its Asylum Case Summarisation (ACS) tool since 1 April 2026. Of those, 203 were first-stage reviews: 139 passed, 19 were flagged as needing further review and 45 failed. That is 45 of 203, or 22.2 per cent (derived). The other 56 were second-stage reviews of summaries flagged or failed at the first stage: 18 passed, 8 were flagged again and 30 failed. The department set the failure rule, ran the checks and supplied the counts, so these are its figures, not ORG's.

Two things sit under that headline. First, 22.2 per cent is a first-stage rate. The 30 second-stage failures are 14.8 per cent of the 203 (derived), and ORG's own press release uses 15 per cent. Eight of the 64 summaries that did not pass first time have no second review in the response (derived: 64 minus 56), and 8 more were flagged again at second stage, so the settled failure share lies between 14.8 and 22.7 per cent (derived). The Register's headline, 'one in five', uses the first-stage figure, although its article also reports the second-stage counts.

Second, the denominator. ORG says the 203 is 0.3 per cent of the summaries generated between April and July. That is 203 divided by 70,060 (derived: the 17,515 cases a month in the Home Office's data protection impact assessment, times four). But the assessment says 17,515 'will be' processed, which is a projection. A written answer of 28 April 2026 says ACS 'has (as of today 27/04) become operational', which shortens the live window to about 3.1 months (derived). And The Independent reported on 8 October that around 7,000 summaries a month are now generated, which it says it 'understands' without naming a source. On those bases the 203 is between 0.29 and 0.94 per cent of the output (derived). Every basis leaves more than 99 per cent of summaries with no recorded check.

What the figures do not establish is most of what a reader would want. They do not say whether the 203 were a random sample. They do not say whether the pass standard catches omissions: the Home Office's written failure rule names factual consistency and overall usefulness, and the scoring form says its consistency score 'does not consider whether relevant or salient information has been omitted'. They do not say how many errors sit in the other 99 per cent, whether any decision was changed by a wrong summary, what the tool does to outcomes, or why decisions slowed. ORG says the slowdown does not prove AI caused it, and the figures cannot show a cause either way.

Who is speaking matters. ORG is a digital rights campaign group that has called for both tools to be suspended and independently audited, and its report is advocacy built on freedom of information responses and ministerial letters. The Home Office deployed the tools and says humans remain responsible for decisions. The Register and The Independent are news reports, used here as pointers. This briefing read the primary records: ORG's 20-page report in full, the Home Office's disclosure responses and two data protection impact assessments, two ministerial letters, the pilot evaluation, written parliamentary answers, the Algorithmic Transparency Recording Standard (ATRS) hub, and the UK legislation and guidance named below. OpenAI sells the model family and Microsoft sells the platform the Home Office says it uses, and both sell to public bodies; their documents are used only to confirm names and dates. No applicant or caseworker is named or described, and every figure is an aggregate that a source prints.

All sources were read between 06:35 and 07:05 BST on Wednesday 14 October 2026. The ATRS hub, the Home Office's written answers since 1 October and The Register's article were re-checked at 10:16 BST and had not changed on the points used here. Pages can change within hours, and several items below are live: ORG's internal review request on the DPIAs, the Home Office's promised ATRS records, and the draft ICO guidance.

The counts, stage by stage, and what is derived

The response of 11 September 2026 (reference FOI2026/10371) answers six questions that ORG put on 29 July about the period from 1 April 2026. The counts below are the Home Office's. Percentages, ranges and ratios are derived here, with the arithmetic in the tables that follow.

Review counts from the Home Office FOI response of 11 September 2026 (FOI2026/10371). Shares are derived.

Stage	Reviewed	Pass, flagged, fail	Failing share (derived)
First stage (primary review)	203	139, 19, 45	45 of 203 = 22.2%
Second stage (secondary review)	56	18, 8, 30	30 of 56 = 53.6%; 30 of 203 = 14.8%
Checks in total	259	203 plus 56, as the Home Office states	Distinct summaries checked: at most 203

All 203 are first-stage reviews. The 56 second-stage reviews re-check summaries that were flagged or failed first time, and in the Home Office's words 'some summaries are checked twice', so 259 counts checks rather than summaries. The second-stage 53.6 per cent is not a failure rate for the tool: it is the share failing among summaries already selected for doubt. The Home Office also says that all summaries flagged or failed at primary review move on to a secondary review. That would send 64 summaries on (19 plus 45), and 56 have been reviewed, which leaves 8 (derived). The response does not say what a second-stage 'needs further review' leads to.

Derived figures and their arithmetic. Inputs are the Home Office counts above; the code is in the briefing's working files.

Figure	Value (derived)	Arithmetic and caveat
First-stage failure share	22.2%	45 divided by 203. This is The Register's 'one in five' and ORG's 22%.
Second-stage failures over all checked	14.8%	30 divided by 203. ORG's report and press release print 15%.
Settled share, either way	14.8% to 22.7%	30 of 203 at the low end. At the high end 30, plus 8 flagged again, plus 8 not yet reviewed: 46 of 203.
Live against pilot	2.46 times, or 1.64 times	22.2 and 14.8 each divided by the pilot's 9%. Not like for like: see the section on the scoring form.
ORG's checked share	0.29%	203 divided by 70,060. ORG rounds to 0.3%. Taking 0.3% of 70,060 gives 210, so the gap to 203 is rounding, not a second count.

What the 203 could be out of. None of these is a count of summaries generated; the live window runs from 27 April to the 29 July request date (94 days, 3.09 months).

Basis for summaries generated	Generated (derived)	Checked share (derived)
ORG: 17,515 a month for April to July, four months	70,060	0.29%
DPIA rate of 17,515 a month, from 27 April	54,129	0.38%
The Independent: about 7,000 a month, four months	28,000	0.73%
The Independent's rate, from 27 April	21,633	0.94%

No source read for this briefing prints a count of summaries generated. The ATRS template, discussed below, has a field for 'frequency and scale of usage', and the Home Office's one published record, for a different tool, fills it with a number. Whether the 203 span all of 1 April to 29 July is not stated: the Home Office said on 29 April that ACS was 'not yet operational beyond pilot deployment', and every shorter live window raises the checked share.

Why 22 per cent of a checked sample says little about the other 99 per cent

A failure rate measured on 203 summaries applies to the remaining tens of thousands only if the 203 were drawn at random from all of them and the failure test stayed fixed. The record does not say how the 203 were chosen. The 22 June 2026 letter from the then Minister for Border Security and Asylum says experts 'continue to dip sample ACS output on a weekly basis'. The Home Office's FOI response says results 'have mainly been discussed verbally' in a working group, and the one interim report, 'Update on ACS summary spot testing as of 2 June 2026', has not been published. 'Dip sample' describes a habit, not a design.

If the sample were random, the arithmetic is short. A Wilson 95 per cent interval for 45 of 203 runs from 17.0 to 28.4 per cent (derived, steps below). That is a statement about the rate across all summaries from the period, so it is the only form in which the 203 could speak for the unchecked rest.

Wilson 95 per cent interval for 45 of 203 (derived). The steps reproduce in the briefing's working code.

Step	Value
Observed share, p = 45 / 203	0.2217
z for 95 per cent	1.96
z squared divided by n = 3.8416 / 203	0.0189
Centre = (p + z squared / 2n) / (1 + z squared / n)	0.2268
Half-width = z x square root of (p(1 - p)/n + z squared / 4n squared), divided by (1 + z squared / n)	0.0568
Interval = centre minus half-width, to centre plus half-width	17.0% to 28.4%

Applied to the unchecked summaries, and only if the sample were random and the standard stable, that interval would put the number failing a first-stage test at 11,876 to 19,818 of the 69,857 unchecked on ORG's basis, and 3,643 to 6,079 of the 21,430 unchecked on the lowest basis (derived). Those are what-if figures. A first-stage fail is also not a wrong decision: the record contains no instance of a decision changed by a summary.

If the sample was not random, the interval has no known meaning. Three ways to be wrong are easy to state as hypotheticals, not findings. A sample drawn from summaries a caseworker had already queried would overstate the rate. A sample drawn from the first weeks, before the prompt was tuned, would overstate today's rate. A sample of short, simple interviews would understate it. The record does not say which, if any, applies.

Sampling arithmetic for a plan that could speak for the unchecked output (derived, simple random sampling from a large population).

Aim	Reviews needed or result	Basis
Estimate the rate within 3 points, near a 22% rate	737	z squared x p(1 - p) / 0.03 squared, with p = 0.2217
Estimate within 5 points	266	Same formula with 0.05
Show, with 95% confidence, that the rate is no higher than 1%, after finding no failures	299	ln 0.05 / ln 0.99, rounded up. The rule of three gives 300
One week of about 12 reviews, 3 failing	8.9% to 53.2%	Wilson interval for 3 of 12: one week cannot separate 9% from 22%
Collect 737 reviews at about 12 a week	62 weeks	203 first-stage reviews over 17.1 weeks is 11.8 a week

The Home Office's dip sample works out at about 12 reviews a week (derived), against roughly 1,600 to 4,000 summaries produced a week (derived, from 7,000 and 17,515 a month). A rolling record of every sample, with the selection method written down, is what would let anyone read the results as more than anecdote.

The pilot comparison has the same difficulty. The Home Office's published evaluation says technical specialists 'reviewed all summaries for accuracy prior to use in the pilot', and that 9 per cent 'were deemed to be inaccurate or had missing information' and were removed. The evaluation prints no count behind the 9 per cent. In the pilot, then, every summary passed a human gate before a decision-maker saw it, and the 9 per cent is the share stopped at the gate. In live use the record shows a weekly sample of output, with no statement that a sampled summary is held back until checked; a sample of at most 0.94 per cent cannot act as a gate for the other 99 per cent. ORG sets 9 per cent against 22 per cent. The ratios are 2.46 against the first stage and 1.64 against the second (derived), but the numbers come from different instruments (the pilot's criterion included missing information, as the next section shows) and from different positions of the check, so the comparison is indicative only.

What counts as failing: the scoring form, and 'passed the accuracy check'

The Home Office disclosed the form its experts fill in (Annex B of the response of 11 September). It has four scored criteria, each from 1 to 5 (coherence, consistency, fluency, relevance), a reason box for each, and a yes or no question: 'does the summary function effectively as a case orientation aid?'. The response states the rule. A summary fails if experts judge it 'not useful as a case-orientation aid' or if its consistency score is 2 or below, and a consistency score of 3 is flagged as needing further review. The form also records the case reference and the reviewer's name, so a per-case record exists in principle; the response says the forms are stored by the AI Evaluation Team in Home Office Digital.

How the stated failure rule treats each score. The rule is from the Home Office response of 11 September 2026; the wording is from the Annex B form.

Score	What the form says	Outcome under the stated rule
Consistency 1	'largely factually inconsistent' with the transcript	Fail
Consistency 2	'multiple factual errors or misrepresentations'	Fail
Consistency 3	'some factual inconsistencies or errors' but 'broadly accurate'	Flagged for further review
Consistency 4	'Minor inaccuracies or imprecisions may be present'	Pass
Consistency, scope	'does not consider whether relevant or salient information has been omitted'	An omission cannot lower this score
Relevance 2	'Important points are missing, obscured, or outweighed'	Scored; not named as a failure criterion
Overall usefulness: No	the summary does not work 'effectively as a case orientation aid'	Fail. The only stated route by which an omission could fail a summary
Coherence, fluency	Structure and language only	Scored; not named as failure criteria

ORG reads this as meaning that a summary that drops material facts can pass if what remains is accurate. For the consistency and relevance scores the form supports that reading: consistency is defined as factual accuracy only, and relevance is scored but is not one of the stated failure criteria. The one route by which an omission could fail a summary is the overall-usefulness question, and the record does not say how often experts used it that way, or whether they were told that missing material facts should count. ORG's reading is therefore supported by the written rule and not settled by the practice. The pilot's wording was different: it counted summaries that 'had missing information'. Whether the live test is narrower than the pilot's is not established, which is why the 9 and 22 per cent should not be set side by side without that caveat.

This is the friendly-name fallacy in its plainest form. 'Passed the accuracy check' sounds like a statement that the summary is right and complete. On the form it can mean that the facts the summary chose to include were not materially wrong: a consistency score of 4 passes with 'minor inaccuracies or imprecisions'. Nothing in the form asks what the summary left out, and the only place an expert can say what went wrong is a free-text reason box, so the record holds no error taxonomy that would show how many failures were omissions, distortions or invented detail. A reader of the summary alone cannot see an omission by construction: the missing fact is not on the page.

'Human in the loop' as a label: what the record shows each time it is invoked

The Home Office's position, as a spokesperson put it to The Independent, is that 'asylum decisions are made by trained decision-makers, not artificial intelligence' and that the tools 'cannot be used in isolation and do not make decisions on claims'. That is true of who writes the decision. It is silent on what the human reads, what a wrong summary costs, and who would notice. The table sets each claim against the department's own documents.

Statements by the Home Office and ministers, set against its own documents.

Stated	The record shows	Source
The summary 'does not replace any part of the asylum decision-making process' (Minister, 22 June 2026)	The ACS DPIA says a process was put in place for Technical Specialists to review and summarise a case before it reached the decision-maker, and that the tool 'aims to automate this summary'. How widely that earlier process ran is not stated.	Letter of 22 June; ACS DPIA section 4.1
Decision-makers 'must continue to read all documents related to the claim' (22 June), and the tool saves 23 minutes a transcript (pilot)	The 7 August letter says the summary lets a decision-maker 'refamiliarise' themselves with a case. Whether the transcript is read in full on a given case is not recorded.	Letters of 22 June and 7 August; pilot evaluation
The tools are assistive and 'do not generate new content' (Minister, 7 August)	The evaluation and the DPIA say a large language model summarises the transcripts. A model-written summary is new text; what the sentence can mean is that no facts are added. ORG says both tools 'can filter, reframe or omit' facts.	Letter of 7 August; evaluation; ACS DPIA section 4.11
'Subject Matter Expert (SME) testing continues after operationalisation' for the policy search tool (written answers, 13 to 28 April, repeated in the letter of 22 June)	7 August: testing was 'carried out during the early development' and 'has now ceased', and no summary of findings was produced. The Independent reports the Home Office saying other periodic expert testing continues.	Written answers; letters; The Independent
'Existing quality control processes are followed' (written answers, 27 and 28 April)	7 August: the decision-quality measure of whether all evidence was considered 'does not include the Asylum Case Summary', and the summary 'is not a quality improvement metric'. Expert dip sampling is the only check aimed at the summary, and its results have 'mainly been discussed verbally'.	Written answers; letter of 7 August; FOI2026/10371
Applicants are informed: the DPIA answers yes, through the Borders, Immigration and Citizenship privacy notice	7 August: 'Asylum seekers are not specifically told AI has been used'. The notice, updated 6 August 2026, does not mention artificial intelligence, a language model or summarisation. No source read for this briefing describes a way for an applicant to see or correct a summary.	ACS DPIA sections 2.6 and 2.7; letter of 7 August; privacy notice

ORG's press release says there is no quality assurance for ACS summaries, while its report records 203 expert checks. The two fit together only if 'quality assurance' means the decision-quality framework, and read that way the Home Office's own 7 August letter agrees.

Three mechanisms could change what happens when a summary is wrong. The pilot's pre-use review was one; the record does not say it continues in live use, and the live check it describes is a sample. The dip sample is another, at no more than 0.94 per cent of output. The third is the feedback inbox, which received seven items between April and July (Home Office response, question 5). Seven items in a window in which experts failed 45 of 203 first-stage reviews is the contrast ORG draws between what experts find and what readers report. The two are not directly comparable, but an inbox depends on a reader noticing an error, and a reader who sees only the summary cannot notice what it left out.

What the record states and does not state

The table sets out, topic by topic, what the sources fix and what they leave open. Where sources conflict, both are given.

Stated and not stated, from the Home Office's FOI responses, DPIAs, letters and written answers, with other sources named where used.

Topic	Stated	Not stated
How the 203 were chosen	A weekly 'dip sample' (22 June letter). Forms are stored by the AI Evaluation Team; each records a case reference and a reviewer.	Whether random or stratified, who chose, and what the interim report of 2 June says.
The failure standard	Fail if not useful as a case-orientation aid, or consistency 2 or below. Consistency 3 is flagged.	Whether omissions failed in practice, how usefulness was judged, whether reviewers agree, and what a second-stage flag leads to.
How many summaries exist	DPIA: 17,515 cases a month 'will be processed'. The Independent: about 7,000 a month, no source named.	A count of summaries generated, how many were opened, and whether each was used. The Independent says use is up to the decision-maker.
Go-live date	'As of today 27/04' (answers of 28 April). 'Not yet fully operational' (answers of the same day). 'Not yet operational beyond pilot deployment' (FOI, 29 April). ORG and The Independent: April.	Which date governs the 203, and whether any of the 203 predate 27 April.
Model and platform	'Built on Azure AI Foundry' (FOI, 29 July 2025). DPIA: 'GPT4'. The Independent: GPT-4. ORG and The Register: GPT-4o.	Model version, deployment type, region, the prompt (FOI refused), and whether a second model critiques summaries in live use.
Effect on decisions	DPIA: the tool 'does not impact the type of decision' a claimant receives. Evaluation: no effect on decision quality in a small pilot.	Any decision changed by a wrong summary, outcomes by nationality or language in live use, and whether a caseworker is told of a failed summary (ORG: unclear).
Speed	Decisions a month and interview-to-decision times (below). The pilot saved 23 minutes a transcript.	A cause for the slowdown, how many decision-makers use ACS, and the effect of a smaller initial-decision queue (The Independent).
Applicants	Not specifically told (7 August). The privacy notice has no AI reference.	Any route to see or correct a summary, and whether notice is given in decision letters.
Publication	DPIAs disclosed in redacted form (10 September). ORG says its complaint to the ICO about training materials is under investigation.	ATRS records for ACS or APS, the equality impact assessments, the prompt, training materials, the 2 June interim report, and the DPIA risk scores.
Independent audit	'The Home Office does not plan for any independent audit at this stage' (22 June).	Any inspection of these tools. The Chief Inspector's June 2026 report covers June to December 2025 and mentions ACS only in a survey line.

Slower decisions: the arithmetic, and why the record does not tie it to the tool

ORG's report says average monthly initial substantive decisions fell about 42 per cent and the mean time from interview to decision rose 29 per cent. Those figures check against the Home Office's response of 23 September 2026 (FOI2026/12221). ORG's reference list labels that response a Ministry of Justice one; the letter is from the Home Office.

Initial asylum decisions and time from interview to decision, from Home Office FOI2026/12221 (23 September 2026). Monthly averages and changes are derived.

Measure	1 April to 31 August 2025	1 April to 30 June 2026
Substantive decisions in the window	43,435 (5 months)	15,130 (3 months)
Decisions a month (derived)	8,687	5,043 (41.9% lower)
Mean days, interview to decision	34	44 (29.4% higher, derived)
Median days, interview to decision	17	20 (17.6% higher, derived)
Refusals as a share of substantive decisions (derived)	61.8%	66.6%

Four cautions apply. The windows differ: five months against three, because the Home Office withheld July and August 2026 as data due for publication, so 'the same period in 2025', as the coverage puts it, is not quite the comparison. The number of decisions a month depends on how many cases are ready to decide, and The Independent reports the initial-decision queue 56 per cent smaller in June 2026 than a year earlier, at 40,168 people, with the appeals backlog up 50 per cent to 90,741. ACS was live for about two of the three months in the 2026 window (derived from 27 April). And ORG itself says the figures do not prove that AI caused the slowdown.

The pilot's 23-minute saving came from two decision-making units not chosen to be representative, with 334 logged test cases against 95 comparison cases, and the evaluation notes that its methods rely on self-reporting. The Home Office said on 22 June that its Analysis and Insight team plans follow-up evaluations 'in due course'; none had been published in anything read for this briefing. The throughput data therefore neither confirm nor refute a time saving in live use. They matter beyond the business case: the DPIA's balancing test says efficiencies will mean claimants 'receiving their decisions in a timely manner', so the risk assessment leans on speed.

Same name, different models: which GPT, which service, and a retirement date

The model is named differently by different sources. The ACS data protection impact assessment, version 1.4 dated 23 October 2025, says 'GPT4 Large Language Model'. The Independent says the system 'uses OpenAi's GPT-4 model'. ORG's January 2026 paper says GPT-4, and its June and October 2026 papers say GPT-4o via Azure AI Foundry; The Register repeats ORG. The Home Office told a freedom of information requester on 29 July 2025 that 'ACS is built on Azure AI Foundry' and that interview transcripts are stored on a platform built on Amazon Web Services; its published evaluation says only 'a Large Language Model'. No Home Office document read for this briefing says GPT-4o. GPT-4 and GPT-4o are different models, so this briefing uses the name each source gives and treats the version as unconfirmed.

Two further details bear on assurance. First, the pilot-stage assessment, which the Home Office supplied as the one for the policy search tool although its body describes the summarisation pilot, says the model was used to 'summarise the transcripts and QA the summaries' and to 'critic the summaries before they are released to the user'. The corresponding sentence in the live ACS assessment ends in a redaction, so whether a model checks a model in live use is not stated. If it does, the humans who check summaries are checking a two-pass machine pipeline, and the 203 reviews measure the combination. ORG has asked for an internal review of that disclosure (acknowledged 28 September, answer due 19 October). Second, the live assessment answers 'No' to whether another party is involved and to transfers outside the UK, while the department says elsewhere that the tool is built on Azure and stores transcripts on Amazon Web Services. The redactions may explain the gap; the disclosed text does not.

Microsoft's published retirement schedule lists three GPT-4o versions, all deprecated or legacy. Version 2024-05-13 retires on 9 December 2026, 56 days from today (derived). Versions 2024-08-06 and 2024-11-20 retire on 14 April 2027, 182 days (derived). Its lifecycle policy says Standard deployments can be upgraded automatically to a replacement at retirement unless the customer sets otherwise, and that retirement dates cannot be extended by exception. Whether the Home Office runs any of these versions, on which deployment type and with which upgrade setting, is not stated.

The point is not a flaw in the tool. It is that the 203 reviews measure one model version with one prompt, and a vendor's lifecycle calendar will change the model under the tool on a date the Home Office does not set. The AI Playbook says updates need 'a quantitative testing and validation process', and the Home Office has not said what that would be here. Microsoft sells the platform and OpenAI sells the model, and a retirement schedule is each company's commercial lifecycle policy, not a safety finding.

UK law and guidance, as read from the primary pages

Each row is what the named page says, then what the Home Office record shows. No row is a finding that the Home Office broke a rule. Where ORG argues unlawfulness, that is the view of an opinion ORG commissioned from three barristers, which a court has not tested, and no Home Office reply to it was found.

UK instruments as read on their own pages on 14 October 2026, set against the Home Office record on ACS and APS.

Instrument, as read	What it says	What the record shows
ATRS (guidance updated 8 May 2025; hub read 14 October 2026)	Mandatory for all government departments, for tools with 'a significant influence on a decision-making process with public effect'. The template has fields for scale of usage, human decisions and review, model performance and impact assessments.	The hub lists 152 records, one from the Home Office (a routing tool, published 17 December 2024). None for ACS or APS. The 7 August letter says records are 'being completed retrospectively'.
AI Playbook for the UK Government (10 February 2025)	Principle 4: 'robust assurance and regular checks of the live tool'. Principle 7: central departments must use the ATRS. Outputs are a 'statistically informed guess'.	The Home Office told Parliament the pilots 'predated the AI Playbook' (20 April 2026). ACS went live 441 days after the Playbook (derived); the policy search tool in July 2025.
Data (Use and Access) Act 2025, section 80, in force 5 February 2026 (S.I. 2026/82)	Replaces Article 22 with Articles 22A to 22D. A decision is 'based solely on automated processing' if 'there is no meaningful human involvement'. Significant decisions need safeguards: information, representations, human intervention, contest.	The DPIA answers No to automated decision-making, and the Home Office says a decision-maker writes each decision. The record does not show the provisions apply; they set the benchmark for a label.
ICO draft guidance on automated decision-making (updated 31 March 2026; consultation closed 29 May 2026)	Human involvement must be 'active and not just a token gesture': a person who can change the outcome, is trained in the system's limits and takes account of the relevant data. 'Using ad hoc spot-checking isn't sufficient' for a decision. Monitoring accuracy 'is not human intervention'.	Applies to solely automated significant decisions, which the Home Office says these are not. It is a yardstick for what a human gate must be, not a finding.
UK GDPR Articles 5, 13 and 16, as amended (legislation.gov.uk)	5(1)(d): 'every reasonable step' to rectify inaccurate personal data. 5(2): the controller must be able to demonstrate compliance. 13: information at collection; 13(2)(f) on automated decision-making applies where Article 22C safeguards are required. 16: rectification, including a supplementary statement.	DPIA: public task as lawful basis, substantial public interest for special category data. Applicants 'not specifically told'. ORG's commissioned opinion says that is 'likely to be unlawful'; that is advocacy, untested.
ICO guidance on AI and data protection, accuracy chapter (under review after the 2025 Act)	The accuracy principle 'applies to all personal data', including AI output, without needing 100% accuracy. After deployment, monitoring frequency 'should be proportional to the impact an incorrect output may have on individuals'.	Monitoring is a dip sample of at most 0.94% of output and the written rule does not name omissions. The Home Office has not said how its frequency was set against the impact.
Public Sector Equality Duty (Equality Act 2010, section 149; EHRC page of 28 June 2022)	Public authorities must have 'due regard' to eliminating discrimination and advancing equality. The EHRC says that means being 'fully aware of' the duty and having 'put this knowledge into practice'.	EIAs are said to exist for APS (30 May 2024, updated 11 March 2025) and ACS (6 January 2026, under review). The 7 August letter said the Home Office was 'still aiming' to publish in August 2026; ORG says they are unpublished, and none was found on GOV.UK.
Chief Inspector of Borders and Immigration, asylum casework June to December 2025 (published 25 June 2026)	Paragraph 8.23: AI use 'was in its infancy'. The policy search tool was in all units by July 2025 but 'its effectiveness and reliability were varied'. Evidence that Copilot and ChatGPT 'had been used inappropriately' in decision letters and interviews, which managers said was addressed.	Not an inspection of these tools, and it predates ACS going live. In a survey of 252 staff, 6 had used ACS and 95 the policy search tool. On whether AI tools gave accurate, relevant information, 72 agreed (28.6%), 21 disagreed (8.3%) and 114 said not applicable (derived shares). ORG asks the Chief Inspector to inspect.
NCSC: prompt injection blog (8 December 2025) and Machine learning principles 3.2	Current language models 'do not enforce a security boundary between instructions and data'. Log enough to identify suspicious activity, potentially the full input and output. Audit use, inputs and outputs.	The Home Office says questions asked of the policy search tool are logged and auditable (22 June). Logging of ACS inputs, outputs and prompt versions is not stated, nor how transcripts, which are untrusted third-party text, are handled as input.

The automated decision-making provisions are worth reading even though the record does not engage them, because they ask whether a human's involvement is meaningful. The ICO's draft answer has five parts: a person who assesses the decision at a point where it can change, can influence the outcome, has authority to alter it, is trained and qualified to understand the system's limits and risks, and takes into account the relevant data and factors. Run the label 'human in the loop' through that list for the summary. The decision-maker has the authority. Whether they are trained in the tool's failure modes (the training materials are withheld), whether they take the relevant data into account (they may have only the summary), and whether a spot-check can stand in for checking each case (the ICO says it cannot, for a decision) are the open parts. The Home Office is not shown to be making solely automated decisions, and the ICO text is a draft.

Article 13(2)(f) matters for a different reason. It requires information about automated decision-making only where Article 22C safeguards apply. On the Home Office's reading the summary tool sits outside that, so the transparency question rests on the general fairness and transparency principle in Article 5(1)(a), the information duties in Article 13(1), and public law. ORG's opinion argues the public-law and common-law transparency point. The ATRS adds a separate obligation: the guidance says publishing a record with redactions explained is preferable to not publishing at all, and the Home Office has published a record for a rule-based routing tool, with a field for human review and one for scale of usage, so the department knows the form.

What to do, in the order worth doing

These are controls an AI-governance reviewer can ask for wherever a model writes the text that a decision-maker reads in a high-stakes process. They come from the sources above, not from a view of what the Home Office should have done. Sample sizes are derived from this case's numbers.

The same gap, seen at an AI vendor rather than a public body, is in this site's briefing on Anthropic's OSS Scanner: an expected accuracy rate for reports sent without human review, and no rate printed for the thousands already sent.

What could not be verified

• ORG's figures. ORG's report is advocacy. Its 0.3, 22 and 15 per cent reproduce from the Home Office's own counts and the DPIA's 17,515, and its 42 and 29 per cent reproduce from the Home Office's FOI2026/12221.
• The Independent's 7,000 a month. It is unattributed and could not be verified.
• The model version. The DPIA says GPT4, ORG and The Register say GPT-4o, and no Home Office document read for this briefing says GPT-4o.
• Withheld or unpublished, so not read. The prompt, training materials, the interim spot-testing report of 2 June 2026, the equality impact assessments, the DPIA risk scores, and the working group's verbal results.
• Date of the counts. The FOI response of 11 September answers a request received on 29 July; whether the 203 stand at 29 July or a later date is not stated.
• How pages were read. Whatdotheyknow and Parliament pages returned a bot check to a plain fetch and were read in a browser tab; nothing was submitted. Word and PDF attachments were read as text or as the images inside them. The Register and The Independent were read on their own pages.
• Not covered. The Home Office's separate plans for AI age estimation of asylum-seeking children, and the planned letter-writing assistant, are different tools.
• Time-stamped. Sources were read between 06:35 and 07:05 BST on 14 October 2026, and the ATRS hub, written questions and The Register were re-checked at 10:16 BST. ORG's internal review request on the DPIAs is due by 19 October, the ICO's guidance is a draft, and the Home Office could publish ATRS records or the equality impact assessments at any time.

The question that exposes the gap

What proportion of the summaries a decision rests on has anyone other than the tool read?

Decision-makers read some of them, and the department says they must also read the transcript, though it has not published how many do. The only recorded checks of a summary against its transcript are the 203, out of somewhere between about 21,600 and 70,100 summaries (derived). The department can replace that range with a count whenever it chooses to publish one.

Key facts