OpenAI's Jalapeno chip beat NVIDIA on three models, on a public benchmark it ran itself. Genuinely strong, still unverified, and a chip it will not sell

What OpenAI published

On 25 August 2026 OpenAI published first benchmark results for Jalapeno, its first custom inference chip (co-developed with Broadcom, per reporting), and framed them as an industry lead in AI inference. The claim is broader than a single number. Across three public models, Jalapeno delivered 1.5 to 1.9 times more AI work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the NVIDIA systems it was compared against, rising to 2.1 to 4.1 times higher performance on highly interactive workloads. It is rated at 700 watts, with sustained power at or below 550 watts on the workloads tested, and OpenAI plans to deploy it in its own infrastructure by the end of the year, with a second and third generation already in development.

This is an aggressive announcement, and unlike a bare press release it arrives with numbers. So the useful thing is to read them properly, which means reading both what they show and how they were produced.

The numbers, model by model

OpenAI tested on InferenceX, a public benchmark from SemiAnalysis that measures the full path of serving a request, across three open models. The appendix is specific, and it is worth seeing in full.

Jalapeno versus NVIDIA, per model, as OpenAI reports it

Model, versus NVIDIA system	Peak mixed tokens/sec per kW	End-to-end latency	Min time between tokens
GPT-OSS 120B, vs GB200 (1,200W)	85,448 vs 44,960 (~1.9x)	1.03s vs 1.80s (~1.7x)	0.69 vs 1.87 ms (~2.7x)
DeepSeek R1 670B, vs GB300 (1,400W)	19,641 vs 11,781 (~1.7x)	1.65s vs 5.99s (~3.6x)	1.43 vs 5.90 ms (~4.1x)
Kimi K2.5 1T, vs GB300 (1,400W)	18,195 vs 11,862 (~1.5x)	1.56s vs 5.31s (~3.4x)	1.44 vs 5.48 ms (~3.8x)

Two honest observations. First, this is a real spread across three models of very different sizes, not a single cherry-picked run, which is more than most first-party benchmarks offer. Second, OpenAI also reports eye-catching figures like around 104 times more throughput at the previous-best time between tokens. That number is real, but it measures throughput at the exact latency point where the NVIDIA system does worst, an operating point chosen to flatter, and it is not the number to quote.

What the benchmark is, and what it is not

An aggressive benchmark, read like a benchmark

InferenceX being a public benchmark matters, and it is to OpenAI's credit that it used one and tested three models rather than a bespoke harness on a single workload. But three things keep this a floor for scepticism rather than a settled result. OpenAI ran the benchmark itself. OpenAI chose the comparison systems, the sequence lengths and the operating points. And OpenAI normalised everything by each chip's published power rating rather than measured power, a choice that favours a chip it says runs well below its 700-watt rating. None of that is dishonest. All of it is why the number to act on is an independent reproduction, not the announcement.

The architecture, in one idea

The design story is coherent, and it explains the latency wins rather than leaving them as magic. Language-model inference has two phases with opposite bottlenecks. Prefill, reading the prompt, is compute-bound. Decode, generating tokens one at a time, is memory-bandwidth-bound, and moving data between chips adds delay that leaves compute idle while it waits. Jalapeno is built to keep model state, including the KV cache, local, and to hold an entire request inside one connected network domain so data barely moves. That locality is the source of the low time-between-tokens figures, and it is a genuine architectural choice, not a process-node fluke.

AI designed the chip, and AI programs it

The part of this post that should interest this site most is not the benchmark, it is the development loop. OpenAI says it went from design to tapeout in nine months by using its own models to explore implementations and optimise the chip's arithmetic circuits, and that with Codex and GPT-Astra it brought three open-weight models to high performance in two months. It also claims AI-generated implementations of selected GPT-OSS attention and mixture-of-experts blocks ran 1.5 to 1.8 times faster than human-expert-written ones.

This is the genuinely new thing. A model family is now being used to design and program the silicon that will serve the next model family, which compresses the hardware development loop and deepens the full-stack advantage OpenAI is claiming. It is also a concentration of capability worth watching, because the same company now owns the model, the serving software, the compiler, the chip, and the tools that program it.

Why it is aggressive, and the supply-chain read

OpenAI is among the largest buyers of NVIDIA's most expensive parts, and it has just argued it can serve its own inference for a fraction of the power on silicon it controls. Publishing now, before an independent number exists, is the point. OpenAI is careful to add that it will keep deploying NVIDIA and other accelerators widely for both training and inference, which is the tell that this is a negotiating position as much as a product.

For anyone building on OpenAI's API, the durable point is that the substrate under your workload is moving from a merchant chip anyone can rent to a stack one vendor owns end to end: model, serving, compiler, chip. That can lower prices and it deepens lock-in, and it is the reseller supply-chain question one layer lower. Not just who serves your prompt, but on whose silicon, designed by whose model.

What to do

The position

The chip looks genuinely good, and OpenAI earned some credit here. It used a public benchmark, tested three models of different sizes, and attached scope to its more surprising sub-claims. An inference-optimised accelerator that keeps model state local and beats general-purpose GPUs on power efficiency is a plausible, even expected result, and if it reproduces independently it changes OpenAI's cost structure in a way that eventually reaches everyone who uses its models.

None of that is settled by a first-results post, which is why the post exists: to set the narrative before an independent number can. The bigger story is quieter than the NVIDIA headline. OpenAI used its own models to design and program the chip that will serve its next models, on a stack it owns from the weights to the wafer. The benchmark is a floor for scepticism. The development loop is the thing to watch.