OpenAI Publishes the First Jalapeño Benchmarks: The Comparison Conditions Matter

Two sets of evidence can test Jalapeño’s commercial value over the next three to six months. OpenAI would need to disclose rack-level power consumption and cost per million tokens, while an outside test would need to reproduce the performance gap using the same models, latency targets, and decoding settings. Without both, the first benchmark establishes speed under a particular configuration but does not establish that production inference is cheaper than on GPUs.

On August 25, 2026, OpenAI published the first results for Jalapeño, its custom inference chip. The company used SemiAnalysis’s InferenceX suite with GPT-OSS-120B, DeepSeek R1, and Kimi K2.5. It reported 1.5x–1.9x the peak throughput of the comparison systems and between 1.7 and 3.6 times lower end-to-end latency. In an ultra-low-latency setting, OpenAI reported a speed advantage of 2.1 to 4.1 times. These are vendor-run measurements, not operating statistics from a large customer deployment.

A Jalapeño rack contains 128 accelerators and provides 1.7 exaFLOPS of four-bit compute, 27.5 TB of HBM4, and nearly 2 PB per second of memory bandwidth. Each accelerator has 216 GB of HBM4 and 15.4 TB per second of memory bandwidth. OpenAI developed the ASIC with Broadcom and optimized it for inference on trained models. Training still requires the broader programmability of Nvidia or AMD GPUs. The company expects initial systems to begin appearing later in 2026, with volume production following in 2027.

Older GPUs and decoding choices narrow the claim

The Register notes that OpenAI compared Jalapeño against Nvidia’s GB200 NVL72 and GB300 NVL72 rack systems, introduced in 2024 and 2025. Nvidia Rubin and AMD MI455X, both expected to ramp in 2027, were not included. Those forthcoming GPUs are also designed for a mixture of training and inference, unlike the specialized Jalapeño chip. The comparison therefore says something useful about the selected systems, but it cannot settle how the custom ASIC will perform against its direct contemporaries.

OpenAI also excluded speculative decoding. That choice can make a hardware comparison cleaner because the benchmark is not helped by a smaller draft model predicting the output of a larger one. Production customers, however, pay for the complete platform. Their relevant question is which system delivers the required output quality at a target latency for the lowest total cost after software acceleration is enabled. A benchmark that turns off a widely used serving technique does not yet answer it.

Power is the other missing input. OpenAI has not disclosed consumption for the full system of 128 accelerators. Memory capacity, bandwidth, and FLOPS describe technical ceilings, while a data center pays for electricity, cooling, networking, software utilization, and failed or underused hardware as well. Jalapeño performs inference only. If workloads do not keep a specialized rack highly utilized, its peak performance advantage will not necessarily translate proportionally into a lower cost per token.

The most informative result before volume production in 2027 would combine energy per token, rack utilization, and failure rates under the same model and service-quality target. SemiAnalysis or another laboratory could then rerun the same InferenceX version on Rubin, MI455X, and Jalapeño with speculative decoding and full-system power included. If the reported 1.5x–1.9x throughput gap survives those conditions, OpenAI will have evidence that specialization produces a durable inference-cost advantage rather than an early benchmark lead.

Sources:

• OpenAI: Jalapeño’s first results show industry-leading speed and efficiency in AI inference
• The Register: OpenAI’s upcoming Jalapeño chip looks like it’ll be an inference beast