Opinion: Why are there so many AI accelerators?

Editor's take: Sometimes, our ideas for new articles stem from work we're doing. Other times, they start with an image or a title. This is one of those posts. We were recently in a pitch meeting with a stealth-mode startup designing an AI accelerator chip. In the middle of that meeting, we saw a vivid image of a computer driving a car – with the accelerator pedal fully pressed – right off a cliff. It was as if our subconscious was trying to tell us something.

The market for AI accelerators is crowded, leaving little room for new entrants. There are now a dozen or so companies in the U.S. designing chips specifically for "AI" workloads. There are a few dozen more in China, and of course, all the hyperscalers have some version of this chip as well.

When we lay out the landscape this way, the problem becomes clear: there are a lot of these chips either on or soon to hit the market. But is anyone going to buy them? Our best guess is that the outcome for many of these startups won't be great. We say this for several reasons.

Editor's Note:
Guest author Jonathan Goldberg is the founder of D2D Advisory, a multi-functional consulting firm. Jonathan has developed growth strategies and alliances for companies in the mobile, networking, gaming, and software industries.

The first is historical. This is not the first wave of AI chip startups – it's the third. The first wave appeared around 2017 when Google unveiled the TPU. There was another wave four years ago, and now a third, spurred by ChatGPT. Almost none of the companies from the first wave still exist today – they were either acquired, shut down, or are lingering in limbo. The second wave was struggling but got a last-minute boost from the hype excitement around ChatGPT. While the third wave could do better, but it still faces the same challenges that hampered the previous two.

The second factor is technical. Simply put, software is moving too fast. We've seen this pattern before. A company unveils its plans for a new chip, and on paper, it offers a significant performance advantage. But by the time the chip is taped out and put into production, the software it was designed to run has changed so much that the chip no longer has a performance edge.

And don't forget, those potential customers are all working on their own chips. Finally, everyone is competing with Nvidia...

The third factor is commercial. Customers are hesitant to try new chips. Porting code to new designs involves significant costs, and no one likes to be the guinea pig. On top of that, there's a limited pool of customers capable of using these chips in data centers. And don't forget, those potential customers are all working on their own chips. Finally, everyone is competing with Nvidia, which sometimes seems unstoppable.

Put simply, there are a lot of companies chasing a small serviceable market.

To be clear, we hate writing this post. We are big advocates for increased venture funding in the U.S. semiconductor industry. And it pains us to criticize chip startups. Notice we are not naming anyone in this piece. Maybe one or two companies can get to scale, and likely many more will be acquired into the big companies who badly need a new approach to their AI designs. And maybe someone gets it right – some combination of technical brilliance, innovative business model and luck – the rewards could be huge, but beyond that this space is likely to prove very challenging.

Herd mentality, but why?

After writing the above, we realized we were left with a question: Why are so many seemingly similar companies chasing a limited market? The answer is not going to win us many friends.

At its core, the problem is that the U.S. venture capital ecosystem has lost its muscle memory for semiconductor investing. We have written a lot about this before. If you forecast out technology industry revenues over the next 10 years, more than 60% will come from hardware, yet over the past decade, only 10% of venture dollars have gone to hardware.

If you forecast out technology industry revenues over the next 10 years, more than 60% will come from hardware, yet over the past decade, only 10% of venture dollars have gone to hardware.

A few years before ChatGPT, we brought an AI chip company to meet one of the best-known Sand Hill Road VCs, a firm that helped fund many of today's semiconductor giants. Despite a warm introduction, it took them a while to find the right person to meet us. Eventually, we sat down with a Valley veteran who had done many of those early semiconductor deals, and he really liked our pitch.

The problem was that by this point, he was an emeritus partner – they had pulled him out of retirement to meet with us – and he no longer had the influence to bring his partners along with the deal.

To be clear, we are fully sympathetic about how this happened. For starters, early-stage investing in semis is expensive. Three people in a literal garage can bootstrap a software company to millions of dollars in revenue. But a chip company needs $50 million just to get its first product out. And for the past 15 years or so, returns on venture semiconductor deals have been paltry.

These difficulties created a negative feedback loop. Junior associates who championed an investment in a chip startup that ended badly remember that experience. By the time they become senior partners, they still carry the scars of that bad exit and avoid the whole sector. If they even still have a job – some of the deals done in the 2000s turned out really badly. Either way, the result is that the major venture funds have gradually shed partners with any knowledge of the sector.

VCs are smart and flexible. They know an opportunity when they see one. And so, past waves of excitement around AI chips, like Google's TPU, which clearly marked an important trend sparked a lot of interest from investors. The problem is that by the time the TPU came around, there was almost no one left in the Valley who had enough semis knowledge to accurately judge the market.

We participated in multiple diligence project in the 2010s where it was clear that the investors did hot have a good grasp of some chip startup's prospects. At times, it seemed like only one firm in the Valley really understood the market, and they were involved in almost every deal.

As a result, we ended up with multiple hype cycles driving investments. The market – and LPs – saw reasons to be interested in chips again, and investors piled in without fully understanding the risks. This process has now repeated itself a few times, but the largely poor exits seen in the sector have only reinforced venture investors' aversion to it.

It's notable that many of the very large AI accelerator fundraises this year have been led by non-traditional venture investors. The best-known firms all got their fingers burned by the first TPU-era wave and have mostly stayed out of the latest deals. Perhaps the best example of this is Sequoia, which has published two pieces cautioning against AI hype. Those notes are partly aimed at Sequoia's own LPs, explaining why the firm is not investing heavily in this space.

The hardest part of all this is that a whole series of rational decisions have led to an undesirable outcome. While we lament the lack of venture investing in semiconductors, we also understand the logic that got us here. We don't mean to criticize established venture investors – they all had good reasons for their actions. Unfortunately, decades of past practices and institutional memory make it very hard to right the ship. It will likely take a new generation of venture firms to get things headed in a more sustainable direction.