Ever since the launch of the “Antares” MI300X and MI300A compute engines by AMD back in early December, we have been mulling over the spending forecasts for AI spending in general and for infrastructure and accelerators more specifically. With the generative AI marketing rocketing upwards on what looks like escape velocity, it is important to temper the great expectations.
During the Antares launch, AMD chief executive officer Lisa Su said that the company was revising its forecasts for the market for AI accelerators in the datacenter – which includes GPUs and other devices and very likely allocates some of the spending on CPUs to AI work as will actually happen in the real world. The revision was eye popping, and it has had us scouring the market researchers of the world for some kind of validation or alternative opinion because of the largesse that AMD said it expected.
To be specific, a year ago, AMD was projecting the total addressable market for datacenter AI accelerators was on the order of $30 billion in 2023 and would grow at around a 50 percent compound annual growth rate through the end of 2027 to more than $150 billion. This sounded like a lot of money, but with the revision a year and a half later, Su said that AMD was pegging the market AI accelerators in the datacenter at $45 billion this year and that it would grow at a more than 70 percent CAGR out through 2027 to reach more than $400 billion.
Now, remember, this forecast is just for the AI accelerators. Not the server hosts, not the interconnect, not the main memory and flash on the hosts, not any software or services for the AI platforms being built. We have been chewing on this, and it still sounds too big to be true. (The IT world could go crazier than it is now, of course, and this model seems to suggest that is indeed what will happen.)
We were grateful this week to see a report coming out of IDC talking about spending on generative AI hardware, software, and services that also included some comments about the broader AI market as well that allowed us to build our own model of what the AI market might look like out to 2027. This model, we openly admit, is a bit like having a piece of dental floss and knitting a six foot scarf out of it. But, in the absence of the full IDC report, we have little choice but to work with what we can find that makes sense and build it out from there.
Here are the data points that IDC gave us all to work with:
Enterprises will invest more than $19.4 billion in generative AI in 2023. We spoke to the press relations people at IDC when we were confused about how the CAGR rates were calculated and learned that IDC believes that $6.8 billion was spent on generative AI hardware, software, and services in 2022. IDC said elsewhere in promotions for its report that the generative AI market worldwide would more than double to $40.1 billion in 2024 and would grow at a CAGR of 86.1 percent to hit $151.1 billion in 2027.
This paragraph in the public statements that IDC made about the global generative AI market is key:
“GenAI Infrastructure, including hardware, Infrastructure as a Service (IaaS), and system infrastructure software (SIS), will represent the largest area of investment during the build out phase. But GenAI Platform and Application Software will gradually overtake infrastructure by the end of the forecast with a five-year CAGR of 99.6 percent. Similarly, GenAI Services, including IT and business services, will nearly equal infrastructure spending by the end of the forecast with a five-year CAGR of 94.2 percent. By the end of the forecast, GenAI spending will account for 29.0 percent of overall AI spending, up significantly from 10.8 percent in 2023.”
We plugged those datapoints into a spreadsheet, as usual, and tried to fill in the gaps, as usual. We do not warrant that these are IDC numbers, of course, except for the ones that are in black ink, and we point out again that any numbers shown in bold red italics are our estimates based on assumptions we make about how the growth rates play out between the endpoint years and how generative AI spending was allocated to hardware, software, and services last year.
We do this merely to try to get a first order approximation of what the generative AI and overall AI total addressable market is. We think we did a hell of a lot better job than any GenAI chatbot would do given the same information.
Anyway, take a look at the table we generated and then we will go through it:
First, if we know the overall GenAI spending and its share of overall AI spending in both 2023 and 2027, we can calculate the overall AI spending levels in those two years. That is just a straight bit of math, which is why those numbers are shown in black. If we assume that the revenue growth rate for AI spending is about the same in 2022 and 2023, we can estimate that GenAi represented about 6.2 percent of all AI spending in 2022, and once we have that datapoint, we can not only calculate that works out to $109.2 billion in spending for all AI in 2022 across hardware, software, and services but also that this would represent a 36.7 percent CAGR over the forecast period to reach the $521 billion in overall AI spending that would be hit in 2027.
Now, let’s talk about the split between hardware, software, and services for GenAI, as hinted about in that big statement above. Our best guess is that GenAI infrastructure represented about $3.2 billion in spending in 2022, with GenAI Platform & Application Software representing $1.9 billion and GenAI Services representing $1.7 billion. With those CAGR figures for the two categories (GenAI software and services, speaking generally), we can calculate where those revenue levels would be in 2027 and remember, we have to get the GenAI software to be bigger than the GenAI hardware in 2027 and the GenAI hardware to be bigger than the GenAI services in 2027.
The numbers we show in our estimates for GenAI hardware, software, and services spending are obviously not the only way to fit these pieces of data together – it is just one possible solution to the simple equations behind all of this. But these numbers pass the smell test, we think. And if IDC’s conception of the GenAI world and our interpolations and interpretations from the data it revealed to the public are “true,” then there will be somewhere north of $50 billion in generative AI hardware sold in 2027 against that overall $151.1 billion in worldwide generative AI spending.
And if the split between hardware, software, and services is similar for all kinds of AI, then of that $521 billion in overall AI spending, then maybe $125 billion or so of that will be for hardware. That is for all hardware – not just for the accelerators as in the AMD forecasts we cited above – and this strikes us as a more reasonable number in a market that will see not only intense acquisition, but increasingly intense competition.
That all presumes that GenAI doesn’t take over and become more dominant across all AI spending. Even IDC puts GenAI at less than a third of overall AI spending.
Anyway, there is your food for thought as we enter the holidays here at The Next Platform. Thanks to you all for reading all these years, and we look forward to a boisterous and interesting 2024. Peace, love, alcohol, fruitcake, sleep, out. . . .
Great sleuthing once more, Timothy.
Reading this, I couldn’t help but wonder, where will the power and cooling come from to run all this AI, and all the non-AI tech we’ll be deploying they 2027.
Will the power and cooling demands put a damper on all this projected growth? I’d love to hear your thoughts on this. After you get some well deserved relaxation in!
I am working on this very thing. The funny version is this: We build a 30 exaflops or 60 exaflops supercomputer to solve the fusion problem and to build a time machine to send a fusion reactor back to 1969, thereby giving us the power to run a 30 exaflops or 60 exaflops supercomputer two decades earlier and we build Starfleet.
Right in time for the last-minute holiday gift-giving shopper! If you have 3 young ones, with $500B at stake, you could do worse than pick toys that orient them individually towards AI services, AI software, and AI hardware, in addition to the more standard piano lessons and platicine (incidentally at its 125th anniversary since Harbutt), so as to hedge bets into their future successes! (eh-he!)
Happy Holidays!
I know I’m not alone when I express a decent degree of skepticism about such projections by the industry. First, they have a vested in pushing this, because it creates a feeling among industry and research that if you aren’t doing GenAI, then you’re ‘falling behind’ (the kiss of death in tech!), and thus it creates a positive feedback loop for them. At the same time, the very real successes, like ChatGPT, are seen as the tip of a huge iceberg, but with a lot of hand-waving about how it’ll transform some distinctly different area – for example, medical imaging will surely benefit from AI (despite earlier challenges with things like Watson), but there are differences between LLMs and, say, cancer-detecting image analysis AI applications.
So where are the growth areas that will put the massive sums into this? And, more importantly, … will it pay off? There are clear application areas that will benefit from GenAI, but there needs to be a return on any such investment, and I’m half inclined to think the promises won’t match the reality.