OpenAI has revealed the first concrete performance data for its custom artificial intelligence silicon, codenamed Jalapeño. During a presentation at the Hot Chips conference on Tuesday, the company shared benchmark results that suggest its upcoming processor could significantly outperform current industry standards in both speed and energy efficiency.
The hardware, which was developed in close collaboration with Broadcom, represents a major shift in OpenAI’s strategy. By moving into custom silicon design, the company aims to reduce its reliance on third-party hardware providers while optimizing its infrastructure specifically for its own large language models. Richard Ho, OpenAI’s head of hardware, described the development as a major step forward for the company’s infrastructure capabilities.
Competitive benchmarks and performance results
The performance claims for Jalapeño are based on the SemiAnalysis InferenceX benchmark, a standard used to measure the efficiency of chips dedicated to AI inference. Inference is the process where a trained model responds to live user prompts, a task that accounts for the vast majority of operational costs for AI companies.
According to the data presented at the conference, Jalapeño registered more tokens per user and higher throughput per kilowatt than the Nvidia Blackwell system, which is currently considered the state of the art in the industry. These metrics are critical for scaling AI services like ChatGPT to hundreds of millions of users without incurring prohibitive energy costs.
Ho noted in a press call that the results demonstrate a very significant performance advance over current technology. He explained that Jalapeño is designed to serve more AI work per unit of power while simultaneously returning responses more quickly. This balance allows the chip to handle a high volume of customers while maintaining low latency, which is essential for real-time applications.
A full stack approach to silicon design
One of the defining characteristics of the Jalapeño project is what OpenAI calls a full-stack approach. Rather than designing a general-purpose processor, the company developed the chip, its AI models, and its memory systems in concert.
The company even utilized its own AI models to assist in the hardware development process. This integrated methodology allowed the hardware team to address specific friction points in the inference process that general-purpose GPUs might not handle as efficiently.
Because OpenAI knows exactly how its models operate, it can tune the silicon to the specific mathematical operations and data movement patterns those models require. This multigenerational platform strategy is intended to ensure that future versions of OpenAI’s models and chips continue to evolve together.
Solving the data movement bottleneck
The technical architecture of Jalapeño focuses heavily on reducing bottlenecks during the prefill and communication phases of inference. In AI processing, the prefill phase occurs when the model first "reads" the user's prompt, while the communication phase involves moving data between different parts of the system as the model generates a response.
OpenAI stated in a blog post that Jalapeño was designed to minimize data movement and communication delays. To achieve this, the system allows for the explicit placement of model state. This includes the KV cache, which is the memory used to store context during the generation of a response. By keeping this data local while activating the necessary combination of compute, memory, and networking for each phase, the system reduces the time and energy spent moving data back and forth across the chip or the network.
This localized approach addresses one of the primary challenges of modern AI: the "memory wall." As models grow larger, the speed at which data can be moved from memory to the processor often becomes the limiting factor, rather than the raw speed of the processor itself.
The road to deployment
While the benchmark results are promising, it will be some time before Jalapeño is powering OpenAI’s production workloads at scale. Ho provided a timeline for the chip’s rollout, estimating that Jalapeño would begin deploying at the end of 2026 in very small volumes.
A more significant deployment is expected to follow in 2027. This timeline highlights the long-term nature of silicon development. By the time Jalapeño reaches wide availability, the competitive landscape will likely have shifted. Nvidia and other hardware manufacturers are expected to release several new generations of hardware between now and 2027, potentially narrowing the performance gap OpenAI is currently highlighting.
The collaboration with Broadcom remains a central piece of this puzzle. As a leader in networking and application-specific integrated circuit (ASIC) design, Broadcom provides the manufacturing and engineering expertise required to turn OpenAI’s architectural visions into physical silicon.
Why this matters for the AI industry
The development of Jalapeño is part of a broader trend among tech giants, including Google, Amazon, and Microsoft, to design their own AI chips. For OpenAI, the motivations are both financial and operational. Custom silicon could allow the company to lower the cost of running models like GPT-4 and its successors, potentially making high-end AI more accessible or increasing the company’s profit margins.
Furthermore, by controlling the hardware, OpenAI can design features that support specific research goals, such as new types of reasoning or larger context windows, that might be difficult to implement on standard hardware.
As OpenAI moves from a pure software and research organization into a vertically integrated technology company, the success of Jalapeño will be a key indicator of its ability to maintain its lead in an increasingly competitive and hardware-constrained market.
Filed under: AI, TechNews, Startups, OpenAI, Broadcom, Semiconductors