Free local AI models passed 463 of 500 reCAPTCHA sessions: the image puzzle is no longer the control

463 of 500, and what that number measures

An automated solver built on two free, openly available vision models passed Google reCAPTCHA in 463 of 500 live sessions, a session success rate of 92.6%, working through 2,931 image challenges on a consumer laptop with no dedicated graphics card and no paid API. That is the headline measurement in Robot Visions: Breaking reCAPTCHA at Zero Cost and Zero Shot, a paper by independent researchers Suphannee Sivakorn and Samantha Gottlieb, posted to arXiv on 16 September 2026 and accepted for the 29th Information Security Conference (ISC 2026).

The number is checkable against the paper's own tables. Table 6 splits the 500 sessions by how many Type B challenges (one photograph cut into a 4x4 grid) each session received. Back-calculating from the published rates gives 217, 21, 187 and 38 successes, which sum to 463. Of the 37 failures, 36 were in sessions where at least half the challenges were Type B.

That distinction is the whole story for defenders. Plenty of organisations describe a CAPTCHA widget as "bot protection" in risk registers and supplier questionnaires. If the widget's hardest step can be passed by a laptop, the question is what else is standing, and whether anyone has checked.

What the researchers actually built

The paper describes two separate solvers, and the difference between them matters because the abstract's two biggest claims come from different ones.

Solver one: local vision models. For Type A challenges (a 3x3 grid of independent images, "Select all images with" a keyword), the researchers use CLIP, an openly available model that scores how well an image matches a text phrase. Each tile is compared against a positive prompt naming the keyword and a negative prompt meaning "something else", and selected if the difference clears a threshold. For Type B (one photograph cut into 16 tiles), they use OWLv2, an openly available object detector that finds objects named in plain text, then map its bounding boxes onto the grid and select tiles with at least 10% coverage. Browser control is Selenium with ChromeDriver, plus what the paper calls standard anti-detection measures: hiding the navigator.webdriver flag, removing automation indicators and adding small random offsets to mouse movements.

Solver two: a prompt-only skill. A structured markdown file of natural-language instructions, run inside Anthropic's Claude Code using Claude Sonnet 5, reads the tile images and picks the answer. This is the basis for the claim that a "non-technical adversary" can defeat reCAPTCHA. It is a commercial, paid model, not a free local one.

The two solvers compared, from Tables 2, 3, 5 and 8 of arXiv 2609.18518

Measure	Local models (CLIP, OWLv2)	Prompt-only skill (Claude Sonnet 5)
Model access	Open weights, run locally	Commercial, paid subscription
Offline Type A, exact match	58% (387 of 662)	95.5% (191 of 200)
Offline Type B, exact match	43.5% (147 of 338)	50.5% (101 of 200)
Cost per challenge	No per-query charge	Mean $0.150 (A), $0.268 (B)
Time per challenge	Under one second with integrated GPU	Mean 38.4 s (A), 83.7 s (B)
Live session success rate	92.6% (463 of 500)	Not reported

The last row is the important one. The paper says single static challenges on Google's demo page "succeeded consistently" with the prompt-only skill, but sessions with dynamic or repeated challenges timed out against reCAPTCHA's two-minute window, and the authors decline to report a formal online success rate "given this latency sensitivity and the small scale of verification conducted". So the free solver has a measured live result, and the no-code solver does not. The skill-floor argument is plausible, and the offline accuracy supports it, but it is the least-measured claim in the paper.

Zero cost and zero shot, defined precisely

Both phrases in the title are accurate within the paper's own definitions, and both are narrower than they sound.

What the title's terms mean in arXiv 2609.18518, sections 1, 3.3 and 4.3

Term	What the paper means	What it does not mean
Zero shot	No training or fine-tuning of model weights on CAPTCHA images; any keyword accepted	No labelled data used: thresholds were tuned on 1,000 human-labelled challenges
Zero cost	No API or per-query charge; runs on a laptop without a dedicated GPU, CPU-only possible	No effort: browser automation, anti-detection and retry logic still had to be built
Locally run	Open models: CLIP ViT-H/14 (632M parameters) and OWLv2	Covers the Claude skill: that runs on a paid commercial service
Zero code	The prompt-only classifier is natural-language instructions	No code at all: the online version wraps it in a Selenium script

The resource figures are modest. The paper's Table 7 gives CLIP ViT-H/14 at 3.7 GB of storage and 4.3 GB peak memory, OWLv2 at 591 MB and 1.2 GB, with CPU-only latency of 2.6 seconds and 1.0 second per challenge respectively. The authors estimate 10 to 15 joules per challenge. At the upper figure, a thousand challenges is 15 kilojoules, about 0.004 kWh.

The paper also notes that its end-to-end pipeline "was generated via AI-assisted coding". That supports the lower skill floor, but a working automation stack still had to be assembled around the models. The authors did not release that online solver.

What was tested, and what was not

Every experiment ran in a fresh browser with no Google sign-in, no browsing history and a new browser instance per session. The authors call this the hardest case for a solver because reCAPTCHA then consistently serves image challenges rather than waving the session through on reputation. That is a sound way to isolate the vision problem. It also means the study measures the puzzle, not the scoring system around it.

Scope of arXiv 2609.18518: what the paper states and what it does not

Question	Stated in the paper	Not stated
reCAPTCHA version	v2 checkbox flow with image challenges	Any separate test of v3 scores, Enterprise or Fraud Defense keys
Challenge types	Type A static and dynamic, Type B	Audio challenge; Google's newer QR code challenge
Where it ran	Official demo page, third-party demo sites, live production login pages	Which sites, how many of each, or which key types they used
Definition of success	Checkmark granted within 20 challenges	Whether the token then passed the site's server-side verification
Network conditions	New browser instance per session	IP addresses, proxies, request rate or geography
Comparison	Prior research: 70.78% (2016), 60.7% across 26 CAPTCHA types (2025)	Human pass rates or paid human solving services

The v3 point needs care. The paper's background section says that when v3 cannot confirm a user, "a visual challenge is deployed as a fallback", and cites a blog post by Friendly Captcha, a company that sells an alternative to reCAPTCHA. Google's own v3 guide describes something different: v3 "will never interrupt your users", returns a score, and leaves the response to the site, for example requiring two-factor authentication or email verification on a low-scoring login. A visual fallback exists only where a site or a key type puts one there. The paper's warning about the fallback is fair for those integrations. It is not a finding about v3 scoring itself.

What Google's documentation already says

Since 22 April 2026, reCAPTCHA has been part of Google Cloud Fraud Defense. Google says existing customers need no migration and keep their site keys. The classic developer pages for v2 and v3 now carry a banner marking them deprecated.

The current key type guidance sets out three web options:

• Score-based keys (Google's recommended option) return a risk score and never show a challenge.
• Checkbox keys show "I'm not a robot" and might show image challenges. Google says it does not recommend them because they increase friction "and don't significantly improve accuracy".
• Policy-based challenge keys trigger image challenges whenever the score falls below a threshold you set.

The same page lists caveats about challenges that read like a summary of the new paper. Google writes that, due to advances in computer vision, CAPTCHAs "are becoming less useful to distinguish between humans and bots", and that they are under threat from paid attackers who can solve all types of challenges.

Two further details from Google's documentation matter more to most UK teams than the paper does.

First, the value of any key type depends on your server checking the token. Google's verification guide says each token is valid for two minutes and can be verified only once, and the assessment guide says to confirm that the returned action matches the action you expected, because a mismatch suggests an attacker falsifying actions. A site that renders the widget but never verifies the token server-side has no protection from it at all, whether or not the image puzzle is solvable.

Second, the classic reCAPTCHA FAQ (now marked deprecated) says that a v3 site key which exceeds its free monthly quota may fail open by returning a static score of 0.9 with the message "Over free quota.", with "no user-visible indications". The quota it describes is 1,000,000 calls per month per domain. A high-traffic site in that state would read every visitor as low risk.

Reading the numbers carefully

The online result is higher than the offline accuracy, which looks odd at first: 92.6% of sessions passed although the local models got only 58% of Type A and 43.5% of Type B challenges exactly right. The paper gives two reasons. reCAPTCHA keeps serving new challenges rather than failing a session on one wrong answer, and it accepts near-miss selections that a strict exact-match test would count as wrong. Successful sessions needed a median of 3 challenges and 57 seconds. Failed sessions are reported as running into the authors' cap of 20 challenges.

Session success by share of Type B challenges, from Table 6 of arXiv 2609.18518; failure counts computed from the published rates

Type B share of session	Sessions	Success rate	Failures (computed)
0% (all Type A)	218	99.5%	1
1 to 49%	21	100.0%	0
50 to 99%	218	85.8%	31
100% (all Type B)	43	88.4%	5
Total	500	92.6%	37

Three internal tensions are worth flagging, none of which changes the central result.

• Why sessions failed. Section 4.2 says failures are "not caused by misclassification" but by reCAPTCHA's risk model sustaining suspicion. The next paragraph calls Type B, the challenge the solver is worst at, "the primary failure driver". The table above cannot separate those explanations. If the risk model is the cause, the scoring layer did some useful work against a clean automated browser in 37 cases.
• Table 5 arithmetic. Failed sessions are listed with a minimum, median and maximum of 20 challenges but an average of 19.4, which cannot all be true.
• Keyword count. The dataset is said to span 13 distinct keywords, but only 11 are named.

The result is also less novel than the headline suggests. A 2024 paper by Plesner, Vontobel and Wattenhofer, Breaking reCAPTCHAv2, reported solving 100% of reCAPTCHA v2 captchas with YOLO models. According to the new paper, the tile classifier in that work was fine-tuned on about 14,000 labelled images covering a fixed set of 13 object classes. The new work's contribution is that no training, no graphics card and no API are needed, and that any keyword works. That is a lower cost and skill floor, not the first break.

The paid route is not cheaper than earlier work either. The Claude skill averaged $0.150 per Type A challenge, 6.25 times the $0.024 per challenge that the paper cites for the 2025 Halligan study using GPT-4o. The authors estimate that a session running to 20 challenges could cost approximately $2 to $13 in tokens.

The friendly-name fallacy: a "bot protection" label on a form

The checkbox says "I'm not a robot". Supplier documentation says "bot protection". Neither phrase is a control. The control is whatever your application does with the verdict, and the paper is a useful prompt to write that down honestly.

The common failure pattern is a policy that turns a low risk score into a picture puzzle, and a passed puzzle into full trust. On the evidence of this paper, that design converts "this session looks automated" into "let it in" for roughly nine automated sessions in ten under the paper's test conditions, at no marginal cost to the attacker. Google's own login advice for low scores is multi-factor authentication or email verification, not a harder puzzle.

Controls on public forms and login pages if the image challenge is assumed solvable. Analysis by pk-sharma.com drawing on Google and OWASP guidance

Control	Still stands	Does not stand
Image challenge as the gate	Adds some delay and friction	As proof that a human is present
Risk score	Not defeated by this paper; still a signal	As a pass if low scores route to a solvable puzzle
Server-side token check	Blocks forged, reused, expired and off-site tokens	Against a genuinely solved challenge
Rate limits and velocity rules	Untouched: a successful solved session took a median of 57 seconds	If limits are per IP only and attackers rotate addresses
MFA, passkeys, breached password checks	Untouched: solving a puzzle does not supply a second factor	Where MFA is optional and most users skip it
Contact and sign-up forms	Moderation, email verification, volume alerts	Any automatic action triggered by a passed checkbox alone

The OWASP Credential Stuffing Prevention Cheat Sheet already treats CAPTCHA as one layer among many, notes that tools and services exist to break them "with a reasonably high success rate", and suggests monitoring solve rates, since abnormally high rates can indicate automated breaking. It ranks multi-factor authentication as by far the best defence against credential stuffing and password spraying. In April 2026 the NCSC announced that passkeys should be the default authentication option businesses offer consumers.

There is also an accessibility cost to answering this paper with harder puzzles. WCAG 2.2 success criterion 3.3.8 treats object-recognition CAPTCHAs as an allowed exception at level AA but not at AAA. Making the puzzle harder would burden people with disabilities and do little against an attacker who retries for free.

Method, disclosure and interests

The authors reported the findings to Google through its Bug Hunters programme on 6 September 2026 (issue 557908167), ten days before the arXiv submission on 16 September. The paper does not record any response from Google, and no public Google response was found for this briefing. The authors did not release the end-to-end online solver. They have published the labelled challenge dataset and the classification prompt used by the Claude skill; this briefing does not link to them.

The researchers list themselves as independent and declare no competing interests. Suphannee Sivakorn is also first author of the 2016 study that reached 70.78% against reCAPTCHA with deep learning, so this is a return to earlier work rather than a new entrant. The experiments included "live production login pages" that the paper does not name. The authors say no user account was targeted and no user data involved.

Commercial interests sit around the paper rather than in it. Its description of the v3 fallback relies on a blog post by a reCAPTCHA competitor. The no-code solver is built on Anthropic's commercial model, and its per-challenge costs are Anthropic's pricing. Google, for its part, is now marketing a QR code challenge as the AI-resistant answer to exactly this kind of attack. None of that undermines the measurement. It is a reason to test claims from any of them, including Google's, before relying on them.

What UK teams running public forms and logins should do

The question that exposes the gap

The paper's honest conclusion is narrow: the image puzzle no longer separates people from software, and it has not for some time. What it cannot tell you is how much of your own defence rested on that puzzle.

So ask the person who owns your login page one question: if every image challenge on this site were passed by software from tomorrow morning, which control would stop the next credential stuffing run, and who would receive the alert?

Key facts