The First Real Numbers
At the Hot Chips 2026 conference on August 25, OpenAI delivered the first independent benchmark results for Jalapeno, its custom AI inference chip co-developed with Broadcom. The numbers were striking: on SemiAnalysis InferenceX benchmark, the 700-watt Jalapeno outperformed Nvidia's 1,400-watt GB300 system across multiple metrics.
Specifically, Jalapeno delivered 1.5 to 1.9 times more AI work per watt at peak throughput and achieved 1.7 to 3.6 times lower end-to-end latency across tests involving OpenAI GPT-OSS 120B model, DeepSeek R1, and Kimi K2.5. For highly interactive workloads, performance improved by up to 4.1 times.
"The bottom line is that the results show a very, very significant performance advance over state of the art," said Richard Ho, OpenAI's head of hardware.
Full-Stack Advantage
Jalapeno's edge comes from OpenAI's full-stack approach. Because the company designed the chip, models, and software together, it was able to optimize for specific phases of the inference process that typically cause bottlenecks — particularly the prefill phase and the communication phase.
The chip is designed to minimize data movement and keep the KV cache — the memory the model uses while generating a response — local to the compute units. This reduces the latency that comes from shuttling data back and forth across a system.
A Narrow But Growing Window
The catch is timing. Jalapeno will deploy in limited volumes at the end of 2026, with meaningful rollout planned for 2027. By then, Nvidia's Vera Rubin platform — which is already in production — will offer substantially improved performance. OpenAI's benchmark comparison is against Nvidia's current Blackwell generation, not the next one.
OpenAI has said it plans to make Jalapeno a multigenerational platform, iterating on the chip, models, and memory in concert. The company also plans to continue using Nvidia chips extensively alongside its own silicon.
The Bigger Picture
Jalapeno is the most visible example of a broader trend: major AI companies are building custom silicon to reduce their dependence on Nvidia and optimize for their specific workloads. Google has its Tensor Processing Units, Amazon has Trainium and Inferentia, and Meta has been developing its own training chips. But OpenAI's results suggest that a company building chips specifically for inference can achieve meaningful efficiency gains over general-purpose GPU platforms.
The question is whether those gains survive the next generation of Nvidia hardware. If they do, custom inference silicon could become a permanent feature of the AI landscape.
