Ongoing Saga: How Much Money Will Be Spent On AI Chips?

Everybody knows that companies, particularly hyperscalers and cloud builders but now increasingly enterprises hoping to leverage generative AI, are spending giant round bales of money on AI accelerators and related chips to create AI training and inference clusters.

But just you try to figure out how much. We dare you. The numbers are all over the place. And that is not just because it is tough to draw the lines that separate AI chippery from the stuff surrounding it that comprises a system. Part of the problem with estimating AI market size is no one really knows what happens to a server once it is built and sold and what it is used for. How do you know, for instance, how much AI work or HPC work a machine loaded up with GPUs is really doing?

Back in December as we were winding down for the holidays, we drilled down into the GenAI spending forecast from IDC, which was fascinating in that it broke GenAI workloads separate from other kinds of AI, and talked about hardware, software, and services spending to build GenAI systems. We, of course, made reference to the original and revised total addressable market that AMD chief executive officer Lisa Su made in 2022 and 2023 regarding datacenter AI accelerators of all kinds, including GPUs but all other chippery.

To remind you, Su said the total addressable market for datacenter AI accelerators was on the order of $30 billion in 2023 and would grow at around a 50 percent compound annual growth rate through the end of 2027 to more than $150 billion. But a year later, as the GenAI boom went sonic as the “Antares” Instinct MI300 series GPUs were launched in December, Su said that AMD was pegging the market for AI accelerators in the datacenter at $45 billion in 2023 and that it would grow at a more than 70 percent CAGR out through 2027 to reach more than $400 billion.

That’s just for the accelerators – not the servers, the switches, the storage, and the software.

Pierre Ferragu of New Street Research, whose team does very smart work in the tech sector, had a stab at how this $400 billion datacenter accelerator TAM might break down, which they tweeted out:

We still think this is a very large number, which presumes somewhere around $1 trillion in AI server, storage, and switching sales at the end of the TAM forecast period.

To come back down to reality a little, as we started out 2024, we grabbed the GPU sales forecast from Aaron Rakers, managing director and technology analyst for Wells Fargo Equity Research and had some spreadsheet fun. That model covers GPU sales in the datacenter from 2015 through 2022 and had estimates to close out 2023 (which had not yet ended when the forecast was made) and stretched out to 2027. That Wells Fargo model also predated the revised forecasts that AMD has made in recent months, where it says it will make $4 billion on GPU sales in 2024. (We think it will be $5 billion.)

In any event, the Wells Fargo model shows $37.3 billion in GPU sales in 2023, driven by 5.49 million unit shipments for the year. Shipments almost doubled – and include all kinds of GPUs, not just the high-end ones. GPU revenues were up by a factor of 3.7X. The forecast is for 6.85 million datacenter GPU shipments in 2024, up 24.9 percent, and revenues of $48.7 billion, up 28 percent. The 2027 forecast is for 13.51 million GPU units shipped, driving $95.3 billion in datacenter GPU sales. In that model, Nvidia has 98 percent revenue market share in 2023 and only drops to 87 percent share in 2027.

That beings us to now. Both Gartner and IDC have recently put out some stats and forecasts for AI semiconductor sales, and it is worth walking through them so we can try to reckon what the AI chip terrain looks like now and might look like some years hence. The publicly available reports from these companies always skimp on the data – they have to make a living, too – but usually say something of value that we can work with.

Let’s start with Gartner. Nearly a year ago, Gartner put out a market study on AI semiconductor sales in 2022 with a forecast in 2023 and 2027, and a few weeks ago it put out a revised forecast that had sales for 2023 with a forecast for 2024 and 2028. The market study for the second report also had a few stats in it, which we added to create the following table:

Compute Electronics is a category that we presume includes PCs and smartphones, but even Alan Priestly, the vice president and analyst at Gartner who built these models, knows that by 2026, all PC chips sold will be AI PC chips because all laptop and desktop CPUs will include neural network processors of some sort.

AI chips for accelerating servers is what we care about here at The Next Platform, and the revenue from these chips – we presume excluding the value of the HBM, GDDR, or DDR memory attached to them – was $14 billion in 2023 and is expected to grow by 50 percent in 2024 to reach $21 billion. But the compound annual growth rate for AI accelerators for servers is expected to be only about 12 percent between 2024 and 2028, reaching $32.8 billion in sales. Priestly says that custom AI accelerators like the TPU and the Trainium and Inferentia chips from Amazon Web Services (to just name two) only drove $400 million in revenue in 2023 and will only drive $4.2 billion in 2028.

If the AI chip represents half of the value of a compute engine, and a compute engine represents half of the cost of a system, then these relatively small numbers could add up to a fairly large amount of revenues from AI systems in the datacenter. Again, it depends on where Gartner is drawing the lines, and how you think they should be drawn.

Now, let’s take a look at how IDC is thinking about the AI semiconductor and AI server markets. The company posted this interesting chart a few weeks ago:

In this chart, IDC is putting together all of the revenues from CPUs, GPUs, FPGAs, custom ASICs, analog devices, memory, and other chippery that is used in datacenter and edge environments. Then, it is taking out the compute, storage, switch, and other device revenues as these devices apply to AI training and AI inference systems. This is not the value of all of the systems, but of all of the chips in the systems; so it does not include chassis, power supplies, cooling, motherboards, riser cards, racks, systems software, and such. This chart has actual data for 2022 and is still estimating data for 2023 through 2027, as you can see.

This chart is a little hard to read, so click it to enlarge it if your eyes are strained like ours are. In this IDC analysis., the AI portion of the semiconductor pie grew from $42.1 billion in 2022 to an estimated $69.1 billion in 2023, which is a growth rate of 64.1 percent between 2022 and 2023. This year, IDC thinks that AI chip revenues – and that does not mean just XPU sales, but all chip content that goes into AI systems in the datacenter and at the edge – will rise by 70 percent to $117.5 billion. If you do the numbers between 2022 and 2027, IDC reckons the AI chip content in datacenter and AI systems will see total revenues on the bills of material growth by a compound annual growth rate of 28.9 percent to $193.3 billion in 2027.

The blog post where this chart came from was published at the end of May and is based on a report done in February. So take that time lag into account.

In that post, IDC added in some figures for server revenues, pulling out AI servers distinct from servers used for other workloads. We did some spreadsheet work on raw IDC server numbers to try to figure out AI server spending back in October 2023, but here is some real data.

In 2023, IDC reckons that the number of servers sold worldwide dropped by 19.4 percent to just under 12 million units, but thanks to the ASPs for AI servers being very high – our best guess is between 45X and 55X that of a generic server for supporting boring infrastructure applications – the revenues for AI servers (based on the estimate for 2022 AI server revenues we came up with last year of $9.8 billion) was up by a factor of 3.2X to $31.3 billion, which is 23 percent of the market. IDC is projecting that by 2027, AI servers will drive $49.1 billion in sales. IDC did not say what its most recent forecast for server revenues were for 2027, but at the end of 2023, that number stood at $189.14 billion as we showed here.

By the way, as IDC carves the server market up, machines that do AI on a CPU using the native matrix or vector engines on those CPUs are not considered accelerated and therefore are not considered “AI servers” in our lingo.

In any event, we think that forecast for AI server revenues for 2027 is too low or the forecast for overall server revenues for 2027 is too high – or a mixture of both. We still think that by 2027, AI servers with some sort of acceleration will comprise a little less than half of revenues and we assume there will be a lot of acceleration and generative AI going on in the datacenter.

But that is admittedly a hunch. We will be keeping an eye on this.