What’s Behind OpenAI’s First Custom Chip Design With TSMC?

The development of AI systems requires vast amounts of computing power, driving unprecedented demand for specialised processors worldwide.

This demand has created a market dominated by Nvidia, whose AI chips power most of the world's LLMs and Gen AI applications.

Yet the resulting dependency on a single supplier has prompted technology companies to seek alternatives, either through partnerships or by developing their own processors.

As the cost of training and running AI models also continues to rise, companies developing LLMs require extensive computing infrastructure, with some projects consuming thousands of processors and the hardware requirements for training models like GPT-4 being estimated to cost hundreds of millions of pounds.

Against this backdrop, OpenAI has reportedly initiated development of its own chips to reduce dependence on dominant supplier Nvidia.

According to Reuters, OpenAI will finalise the design of its first custom chip in the coming months, with plans to manufacture it at Taiwan Semiconductor Manufacturing Company (TSMC), the world's largest contract chip manufacturer.

OpenAI and Broadcom collaborate on processor development

The chip development programme is led by Richard Ho, who joined OpenAI more than a year ago from Google, where he headed the search company's custom AI chip initiative. 

Richard’s team has doubled in the past months in collaboration with Broadcom.

However, Reuters says that OpenAI’s team is smaller than the large-scale efforts at companies such as Google and Amazon – and a new chip design for a large-scale program could cost US$500m for just one version of a chip, with costs potentially doubling when including necessary software and peripheral systems.

This means that OpenAI may need to expand its team significantly for its custom chip to be successful.

TSMC to manufacture using advanced process technology

OpenAI's processor will reportedly be manufactured using TSMC's 3-nanometer process technology, referring to the size of transistors on the chip and the design incorporates a systolic array architecture with high-bandwidth memory, a configuration also used in Nvidia's processors, along with networking capabilities.

The initial chip will support both training AI models and inference, which refers to running existing models.

However, sources indicate the processor will be deployed primarily for inference operations on a limited scale within OpenAI's infrastructure.

Companies increase AI computing investments

OpenAI’s development comes as technology companies increase investment in AI infrastructure, such as Meta planning to spend US$60bn on AI systems in the next year, as Microsoft has announced US$80bn in AI infrastructure spending for 2025.

Meanwhile, OpenAI's participation in the US$500bn Stargate infrastructure programme, announced by the US government, further demonstrates the scale of investment in AI computing resources.

Chip development faces manufacturing challenges

Successful development of OpenAI’s processor would require multiple stages.

For instance, a typical "tape-out" – the process of sending a first design to a chip factory – costs tens of millions of pounds and takes approximately six months to produce a finished chip; yet this timeline could be shortened with additional investment in expedited manufacturing.

The initial processor may require multiple iterations to achieve functionality and if successful, mass production could begin at TSMC in 2026, enabling OpenAI to test alternatives to Nvidia's processors later this year.

Other technology companies have faced challenges in developing custom chips, such as Microsoft and Meta's efforts to produce satisfactory processors having encountered difficulties despite years of development.

Yet recent market developments, including Chinese AI company DeepSeek's announcement about reduced chip requirements for AI model development, have raised questions about future processor demand.

Explore the latest edition of AI Magazine and be part of the conversation at our global conference series, Tech & AI LIVE. 

Discover all our upcoming events and secure your tickets today.

AI Magazine is a BizClik brand