Datacenter Infrastructure Spending Is Up, And Forecasts Are Even Higher

The AI boom is going sonic, and it looks like we had all better cover our ears if we want to be able to hear by the end of the holiday season if the prognostications of the box counters at IDC are correct.

IDC has juts tallied up datacenter spending for compute and storage for the entire world and then diced and sliced it across industry sectors and whether or not the buyer of this gear is deploying it in a cloud for resale or for internal use or buying more traditional so-called non-cloud systems for back office and basic infrastructure services like Web, application, and database services. We think it is very hard to try to draw these lines, but we do like taking a hard look at this IDC dataset to try to figure out if the non-AI portion of the market is in recession.

Spoiler alert! It sure looks like it for five of the six past quarters, including the second quarter of 2024.

Let’s go through the numbers. Take a look at this chart that we built from the historical data that IDC provides for combined datacenter compute and storage spending:

Just a reminder about this data in the table above and the charts and tables below. IDC looks at the money spent on dedicated iron that will be housed in proprietary datacenters or co-lo facilities and amortized over time. It also tracks spending on gear that goes into shared clouds (where others will rent a slice of the machine) or in a dedicated machine (which can be located in a datacenter, co-lo, or cloud and where only one customer occupies a machine). These figures are for factory sales of machinery to the end customers or to channel partners, not the rental income from the cloudy infrastructure.

It takes a while to compile this data, which is why we are talking about the second quarter of 2024 when the fourth quarter has just begun.

In the quarter ended in June, there was a huge boost in spending for machinery that will be deployed in the cloud, what IDC calls a shared cloud to be distinct from on-premises iron sold using cloud-like utility pricing (HPE GreenLake, Dell APEX, and so on) and dedicated bare metal iron bought by clouds to sell that way. Spending on compute and storage for shared clouds rose by a stunning 74.9 percent to $35.3 billion. Before you get all excited by a recovery in server and storage spending, IDC notes that unit shipments for servers and storage arrays were off 17.7 percent. (It did not provide a number.) And added that revenues exploded mostly because of “the exponential increase of GPU server shipments.”

We have been saying for nearly two years that there is an underlying recession for spending on general purpose servers and storage, and for fun we plotted out the unit shipment declines in the table above, which covers the time when OpenAI’s ChatGPT came onto the scene at the end of 2022, starting the GenAI wave that has made Nvidia fabulously wealthy.

Spending for systems going into dedicated cloud infrastructure were only up 5 percent to $6.7 billion in Q2, which dragged down the rate of overall cloud infrastructure spending to only 61.5 percent, pushing it up to $42.9 billion in the quarter. Interestingly, non-cloud infrastructure spending rose by 41.4 percent to $19.4 billion, and we think that some of this is a return to spending for back office systems as well as for private AI clusters that are also bare metal supercomputers in many cases. (Like we said, it gets hard to draw these lines.)

Add it all up, and global datacenter infrastructure spending rose by 54.7 percent to $62.3 billion in the second quarter of this year.

We have been advocating that IT shops looking to free up power and cooling for AI training and inference systems should go as reasonably top bin as possible with their server purchases so they can do the most server consolidation on their general purpose server fleets. (Not the toppest bin parts with highest prices and core counts, but maybe the N-2 products that have most of the cores but not nearly as high of a price.) Some of the server recession as expressed in reduced unit shipments might be due to this activity, particularly for companies whose workloads are only growing at 2X to 3X of gross domestic product and even more true for those who are not even seeing this much growth. For them, it is a straight up consolidation play, and in Europe, the savings in power, cooling, and space can pay for the upgrade over a couple of years.

The way IDC dices and slices its infrastructure spending reports these days, it also lumps hyperscalers, cloud builders, telecom companies, and other niche service providers into a category it calls service providers, who accounted for 67.2 percent of compute and storage spending in Q2 2024. That worked out to $41.8 billion, up 64.2 percent. The remaining $20.5 billion in spending was done by enterprises, governments, and academic institutions, a respectable increase of 32.8 percent compared to the year ago period.

Which brings us to the near-term and long-term forecasts for datacenter infrastructure spending.

Late last year, IDC was forecasting for 2024 spending to acquire shared cloud infrastructure would be $95.3 billion, but in early summer that forecast was raised to $108.3 billion. And now, the forecast has been revised upwards, thanks to AI server spending mainly, to $131.9 billion and increase of 1.58X compared to the spending level in 2023 for shared cloud compute and storage. Dedicated cloud spending is now expected to be $32.1 billion, an increase of 20.4 percent and a few billion bucks higher than originally forecast. That puts overall cloud infrastructure acquisitions up 48.8 percent to $164 billion in 2024. Even non-cloud iron is going to see an 11.7 percent increase to $67.5 billion, rather than a slight decline compared to the original forecast for 2024 back in the fall of 2023.

The spending in the second half of 2024 is going to be particularly large for shared cloud infrastructure, if you do the math on the IDC data. If you subtract out the first two quarters of 2024 from the full-year 2024 forecast, and then compare across the second halves of the prior four years, this pattern is indeed unusual. Spending on shared cloud infrastructure tends to grow around 20 percent, give or take a few points, but this time around, in 2H 2024, it is going to grow by 70.4 percent to $70.3 billion.

Between 2023 and 2028, the current IDC forecast shows spending on compute and storage for cloudy infrastructure will grow at a compound annual growth rate of 18.1 percent to reach $253 billion in 2024. By this time, cloud is expected to comprise a 76.4 percent of overall datacenter compute and storage spending worldwide. Shared cloud stuff will be $198.8 billion of that $253 billion by the end of the forecast period. Spending on dedicated cloud compute and storage (in the several forms we outlined above) will grow at a 15.3 percent CAGR to hot $54.3 billion. And lest you think non-cloud infrastructure is in decline, it is not. The expectation is for non-cloud compute and storage to have a 5.3 percent CAGR and reach $78.3 billion in 2028.