Your token counter may be reading your prompt

What happened

A token counter is the sort of utility nobody thinks about. You paste a prompt, it returns a number, you move on. The mental model most people carry is a pocket calculator: local, inert, uninteresting.

I tested two of the better-known ones on 3 August 2026. The behaviour differs from the mental model, and it differs in opposite directions.

pricepertoken.com/token-counter transmits your text. Typing into the box fires POST /api/tokenEstimation with the raw contents in the request body. I confirmed it by putting a unique marker string into the field and reading it back verbatim from the outbound request, three separate times in a single keystroke's worth of activity. The page also loads PostHog with dead-click autocapture and surveys, Google Analytics, a Cloudflare challenge and a newsletter embed.

token-counter.dev does not transmit your text. It loads a tiktoken WebAssembly module and counts locally, exactly as it says. Its claim that "your text never leaves your device" appears to be true. What is not true is the sentence next to it: "No server, no logs, no tracking." The page loads and executes a third-party analytics script from app.pageview.app, which is why window.plausible exists on the page.

POST https://pricepertoken.com/api/tokenEstimation

{
  "text": "CANARY-7f3a-DO-NOT-SEND",
  "model": "..."
}

Why does it matter?

Notice which one is dishonest, because it is not the one you would predict.

The tool that sends your prompt to a server never claims it does not. It makes no client-side promise anywhere on the page. It is not lying to you. It is simply a server-side tool in a category where users assume otherwise, and it says nothing to correct the assumption.

The tool that keeps your text local is overstating, but on a narrower point: it tracks page views while saying it does not track. Your prompt is safe there. Its privacy copy is still wrong.

The practical consequence is that the marketing copy carries almost no signal. A confident privacy claim did not predict good behaviour, and the absence of one did not predict bad behaviour on the thing that mattered most. If you have pasted a production system prompt, a client transcript, or an internal document into a token counter to size it, the question of where that text went is not answerable by reading the page.

This is the same shape as the shadow AI problem, one layer down. Nobody files a change request to visit a token counter. It is not in any inventory. It sits in the same category as the online JSON formatter and the regex tester: utilities that quietly receive real production data because they feel like local software.

Check any tool in thirty seconds

Why this generalises past token counters

A counter is a small tool and the stakes look small, which is exactly why it is a useful case. The same shape appears wherever a browser tool accepts text and returns something derived from it. Formatters, diff viewers, regex testers, JSON validators, decoders, summarisers. Every one can be built either way, most give no indication which, and the ones handling the most sensitive input are often the ones reached for in a hurry.

Two further points, both demonstrated by the pair tested here. A privacy claim is not evidence, and a page can be accurate about one thing while wrong in the sentence beside it: the tool that genuinely counted locally also stated it loaded no analytics, while loading a third-party analytics script. And the absence of a claim is not evidence either. The tool that transmitted every keystroke made no privacy claim anywhere, which is at least not a lie.

If a tool handles anything that would matter in a breach, the safe default is to assume transmission until the network tab says otherwise.

What to do about it

• Check before you paste, not after. Open developer tools, watch the Network tab, type a few characters into the tool, and see whether anything leaves. This takes about thirty seconds and settles the question permanently for that tool.
• Use a unique marker. Type a nonsense string you can search for, then inspect any outbound request bodies for it. This distinguishes a tool that sends your text from one that merely reports usage telemetry, which is a materially different exposure.
• Treat the absence of a privacy claim as an answer. A tool that processes text locally almost always says so, because it is the main thing worth advertising. Silence is weak evidence of a server round-trip.
• Put the obvious utilities in your AI acceptable use policy by name. Token counters, formatters, diff tools, summarisers. Staff will not think of these as AI tools or as third-party processors, because they do not feel like either.
• For anything genuinely sensitive, count offline. The tokenisers for the OpenAI models are open source and run in a few lines locally. Anthropic publishes no public tokenizer, so any tool claiming to run "the official Anthropic tokenizer" is describing something that does not exist; the authoritative count comes from the count_tokens API endpoint.