Sweetened IT Spending Forecast Is Not Precisely A GenAI Boom

Here is a paradox for you: Spending on infrastructure to support generative AI is apparently booming, as clearly evidenced by the skyrocketing revenues and profits of Nvidia. But spending on datacenter hardware is not changing all that much, and where spending is now forecast to be higher, it is in services and has more to do with offloading other kinds of work than finding GenAI hired guns to do the heavy lifting.

The exceptions to this, of course, are the hyperscalers and cloud builders, who very clearly shelling out tens of billions of dollars each quarter to build massive AI training and now inference systems. But it is important to remember that these companies are not the entirety of the global economy, even if they do, as a group, have a dominant share of the spending in the overall IT sector. That other half of spending is still doing proofs of concept and early deployments for GenAI, and they are taking it slow.

Our thinking on the GenAI boom has been reinforced after reviewing the latest forecast for IT spending coming out of Gartner, and reviewing the stats that the company gave out in a webinar aptly titled, IT Spending Forecast, 1Q24 Update: GenAI – Where is All the Money?

That is what we have been asking for more than a year, and the answer is obviously: “Nvidia has it all. Next question, please. . . .”

But seriously, our sense of this, based on data out of both IDC and Gartner for the past year on server and storage spending, is that aside from the Internet behemoths, IT organizations have been deferring spending on more generic IT infrastructure to come up with the dough to invest in GenAI. And while we are thinking about it, the hyperscalers and cloud builders are doing the same thing. And so overall spending on servers, storage, and switching – the Three Ss of the Glass House, unless you count those additional three Ss in which case there are six – has not had a big GenAI spike. In fact, AI servers loaded up with GPU accelerators have been propping up overall server average selling prices, and therefore overall server revenues, for years. So it was an illusion that the core server business that runs basic infrastructure – print, file, web, application, and database servers in every company on Earth – was healthy. It has not been. Neither OEMs nor ODMs can make very much in the way of profits in this cut-throat manufacturing sector.

The most recent IT spending forecast from Gartner, which was just released but which was completed at the end of March, doesn’t show a Dot-Com Boom spike in server spending. See for yourself:

There has been some change in the forecasts and actual datacenter systems spending in 2023 and the forecast for 2024, and we absolutely think this reflects the increase in AI spending among the hyperscalers and cloud builders.

Let’s go through the numbers. In July 2023, Gartner said that based on its casing of the IT vendors, there was $221.2 billion in datacenter systems spending in 2022, and that this spending would drop by 1.5 percent to $217.9 billion in 2023 and forecast that spending would grow by 8.1 percent to $235.5 billion.

In January this year, a new casing of datacenter system sales was announced for 2022 and 2023 and a new forecast was put out for 2024. Gartner now estimated that sales of servers, storage, and switching in the aggregate would be $226.9 billion, rose by 7.1 percent in 2023 to $243.1 billion, and would grow by 7.5 percent to $261.3 billion. That is an increase in spending of $25.2 billion in 2023 and an increase of $25.8 billion in the 2024 forecast. But if you didn’t track each update to quarterly and annual sales of IT stuff, you wouldn’t see that bump. Which is why we do it.

In the April forecast just announced, Gartner now says that sales of datacenter systems was a little cooler than it thought, at $236.2 billion, up only 4 percent, but expects for sales to grow by 10 percent to $259.7 billion. So, yes, that is 10 percent growth, but it is actually a slightly smaller number than the January forecast in the absolute and all of that additional 2.5 points of growth is coming from the lowering of the sales level in 2023. In the absolute, the new forecast has $6.9 billion less in datacenter systems sales in 2023 and $1.7 billion less in 2024. And we think this represents the worsening of the recession in server spending outside of AI servers.

Let’s take a look at the longer-term trends in IT spending and then drill down into GenAI spending as Gartner sees it.

Here is a monster table that includes all of the publicly available data from Gartner on IT spending for the past dozen years and the forecast for this year:

In this table, the Devices category means PCs, tablets, and smartphones, and telecom services include enterprise spending worldwide for voice and data services. Application and system services for all service providers, including the telcos and cable operators, are in the IT Services category, as are the for-fee IT services sold by the hyperscalers and cloud builders.

To make this data more interesting and relevant, we have pulled out what we call Core IT revenues, which is Datacenter Systems plus Enterprise Software plus IT Services, and we have gauged their growth against global gross domestic product (GDP) growth in each year. We have also calculated the share of total IT spending represented by Core IT revenues.

As you can see, with few exceptions Core IT grows faster than global GDP, and sometimes considerably faster. Perhaps more significantly, the share of total IT spending represented by Core IT has grown from a little more than a third of the IT spending pie to a little more than a half of the pie in a mere dozen years, and it probably will not be long before it represents two thirds of the pie.

Here is how growth for global GDP, Core IT spending, and overall IT spending stack up, visually:

This chart shows a pretty plot of how overall IT spending has grown and how the three segments of Core IT have grown:

For all of the excitement about datacenter gear, this is a very tough line to move up and to the right, and software and services just keep rising and rising as IT infrastructure gets more complex and outside help is needed to get things done, either in the form of packaged software or hired consultants.

And finally before we get into the GenAI effect, here is what Datacenter Systems spending over those thirteen years looks like in the raw and after you inflation adjust it:

That 2018 spike in server, storage, and switch spending is a few things combined: a burst in spending by hyperscalers and cloud builders for generic infrastructure, the first wave of big AI spending by these companies, and an enterprise upgrade cycle that was long overdue. During the coronavirus pandemic, IT spending rose even after adjusting for inflation, and this is the next wave of AI hitting. We think that more than all of that increase in Datacenter Systems spending is due to AI servers, and the evidence from elsewhere bears this out.

According to John-David Lovelock, who is a distinguished vice president and analyst at Gartner, the forecast is for $60.8 billion in AI servers to be bought in 2024. The hyperscalers – what we call the hyperscalers and the biggest cloud builders – will account for 57.6 percent of the pie and other, smaller service providers will account for 11.5 percent or $7 billion. Enterprise will comprise $10 billion in AI server spending, or 16.4 percent, and there is another $8.8 billion, or 14.5 percent of the pie, being spent by other organizations – we presume startups, HPC centers, and such.

Based on the chart that Lovelock showed, it looks like AI server spending by enterprises is expected to nearly double from 2023’s level – call it $4.5 billion or so, and service providers are seeing somewhere around a 1.4X growth from 2023 and hyperscalers are looking at maybe a factor of 1.6X growth from 2023. If you work that backwards, then AI server spending grew by just under 2X – call it 1.95X – from the $31.5 billion level in 2023.

Be careful extracting those numbers out of the Datacenter Systems numbers from Gartner for 2023 and 2024. You don’t know the switching and storage components for AI systems. But because we can’t leave anything alone, if you assume that the average network for an AI cluster is about 15 percent of the total hardware cost and storage is around 10 percent, then AI datacenter systems spending in 2024 will be around $81 billion, or around 31 percent of the Datacenter Systems pie.

Now further, if you assume the same proportion of switch and storage spending for 2023 for AI clusters, then AI system spending in 2023 was around $42 billion, representing around 18 percent of the total Datacenter Systems spending across all workloads and system types. That means AI server spending is expected to nearly double this year. (Remember, this is our math, not Gartner’s math.)

These numbers do not insult our sensibilities. And they roughly map to the shape of the curves of the AI server spending breakdown we extracted from some IDC server data back in October 2023.

Let’s keep going.

We think that spending on AI datacenter systems was relatively small in 2022, maybe one-quarter the size of 2023. And if you do the math on the generic server part of the market, then aggregate revenues from non-AI datacenter systems fell by 10.3 percent in 2023 to $192 billion, and given all of these numbers above, that also implies that non-AI datacenter systems revenues will fall again in 2024 by another 8 percent to $179 billion.

So, that’s where all the money went, and that is how the world is paying for GenAI projects. But it is not like there is a heavy amount of normal IT spending and then exuberant AI spending on top of that like we had during the Dot-Com Boom, which coincided with the Y2K crisis that caused a whole lot of redundant system buying in 1998 and 1999, adding fuel to the fire.

Aside from the hyperscalers and cloud builders, people are being more sensible now and keeping servers, storage, and switches in the field and on the balance sheets longer and seeing where this GenAI wave might lead before they jump on it and spend big bucks.