You know the world is a different place when shipping 2.58 million servers in a quarter feels like a slowdown, a disappointment, and perhaps a leading indicator of an overall economic slowdown in the world.
But the slight 5.1 percent downtick in server shipments in the first quarter of 2019 is probably more a function of the hyperscalers and cloud builders having spent a fortune on infrastructure in late 2017 and early 2018 and then waiting to see what the future holds for future processors from Intel and AMD.
In the quarter ended in March, according to the box counters at IDC, those 2.58 million server shipments drove a 4.4 percent increase in revenues, to $19.8 billion, and the fact that companies are buying increasingly rich configurations, with more main memory, lots of flash, and occasionally GPU or FPGA accelerators, is doing its part to keep the money pile growing even if it is possibly not doing as much as we might hope to increase the profitability of bending metal to build servers.
As far as we can tell, none of the OEMs or ODMs are rolling in black ink, much less swimming in it. But with the average selling price of a server falling through the 2000s, it has to have been a welcome thing to have it rising again through the 2010s. Here is what it looks like for just the X86 portion of the server space, based on IDC data, since the first quarter of 2009:
This is pretty much the mirror image of the decline in server average selling prices from 1999 through 2009 as proprietary and RISC/Unix systems fell out of favor – they used to represent half of system revenues in the mid-1990s, remember, and were the default iron for the dot-com boom until both X86 processors and Linux and Windows Server matured.
We start tracking the modern era of servers as the Great Recession was getting rolling in earnest in 2009. Coincidentally, that was when AMD was backing away from servers after some issues with its Opterons and Intel was, in essence, cloning many of the ideas of the Opterons in its “Nehalem” Xeons. That recession, plus the maturation of server virtualization, plus Intel’s delivery of a good 64-bit, multicore processor combined to put Intel’s Data Center Group on the path to a huge revenue expansion and a jump in profits that was bolstered significantly by the lack of competition.
There are many effects of the Great Recession on the systems business, but one of them is that it relegated all non-X86 – really non-Intel – processors to niche status. The RISC/Unix vendors just could not offer the same bang for the buck as a Xeon system running a virtualization stack and a Linux or Windows Server operating system. At the same time, the hyperscalers and cloud builders had an order of magnitude increase in their compute capacity needs as online applications and cloud computing became normal. So the relative capacity sold each quarter went through the roof, even as the core counts and instructions per clock cycle kept increasing with every processor generation and across every processor.
If you make some assumptions about how many cores were sold each quarter in the average server counted by IDC and layer in the IPC for the chips, you can come up with a relative aggregate performance capacity metric (in our case based on raw integer performance, excluding floating point math) for each quarter; we have the revenues from servers sold directly from IDC, and if you match these two lines up, it tells the story of just how much more capacity the world is consuming, relatively speaking, over the past decade. And it also shows how the price/performance increased dramatically as RISC/Unix machines were either vanquished from the market or dropped their prices to better compete.
A few things to note. What is immediately obvious is that every now and then, the market buys a lot of capacity and then cuts back a little bit. These events are not just tied to CPU product cycles, but they are related. What is also obvious is that the drop-off in capacity from Q4 2018 to Q1 2019 is the largest magnitude downdraft the server market has experienced in the past decade, but it really represents a return the normal slightly exponential but mostly linear capacity curve we have seen over the past decade. The sequential drop is, in fact, 4.4X larger than all of the server capacity sold worldwide in the first quarter of 2009, just to give that some perspective. The hyperscalers and cloud builders pulled back in the third and fourth quarters of last year, but enterprises were still pushing, and when enterprises and hyperscalers and cloud builders all pulled back in the first quarter, it caused that dip. But it is more like the hyperscaler and cloud builders were buying ahead of the launch curve, to be honest, and this is return to “normal.”
The thing is this: The world does a hell of a lot more computing than it used to as more and more aspects of our personal lives – and the corporations that ultimately support those lives – are computerized with all manner of telemetry and data.
The other neat bit about this chart is that it shows that even though the relative price/performance of the average raw server came down a lot between 2009 and 2011, it started petering out then and since Intel has had hegemony of the datacenter, the relative cost of a unit of raw compute in these servers has not come down. Sure, Intel is cramming a lot more raw integer in the machines, memory speeds are increasing to balance it, and flash is replacing disk to keep memory fed, and vector engine performance has grown even faster, but that raw underlying integer performance costs about the same today as it did three years ago. That is, however, an order of magnitude cheaper than it was when the Great Recession was roaring, so there is that. The curve is starting to bend down in the past two quarters, and with renewed competition from AMD, the core counts are going to go up and the prices are going to come down and set a new level a bit lower. And as far as we can tell, all of this really means profits will fall as those prices come down. The question with the future “Rome” Epyc processors from AMD and the “Ice Lake” Xeons from Intel is: Will the volumes expand fast enough so they can at least tread water with absolute profits even as the profit per server chip goes down?
Either way, there is little question in our minds that the days of 50 percent operating margins in server chips are over, unless AMD and Intel collude to keep pricing artificially high. Which is extremely unlikely and also highly illegal. Those profits will never come back, just like they didn’t when RISC/Unix jumped from the scientific workstation to the datacenter and lowered the bar competing with proprietary machines or when X86/Linux and X86/Windows jumped from the desktop and lowered the bar again against RISC/Unix iron. Arm is supposed to be spearheading then next bar lowering, jumping from our smartphones to the datacenter, but this has not quite happened yet. . . .
Money is how we keep score in business, and all we can see from the IDC numbers is what they reckon the factory revenues coming out of the OEMs and ODMs are; we have no idea from the IDC data if any of them are actually making money, and we suspect that in many cases, the OEMs and ODMs are bringing in a few pennies on the dollar for all their efforts and perhaps making more profits with services and sometimes software that they add to their machinery. It’s hard to say, even when looking at the financial results from public companies that play in the server racket. There’s a lot of nonsense in the numbers.
Here is how the vendors stacked up against each other in the first quarter of 2019 and the four quarters of 2018:
IBM, by the way, did not even make it to the top five vendors this time around as its System z14 upgrade cycle is already winding down and even despite growth in its Power Systems business. Selling off that X86 server business to Lenovo really did not help IBM’s systems cause, or its profitability, as it turns out. IBM is still not making as much money in systems as we think it thought it could, and for the reasons we suggest: Prices come down, and profits come down faster, when there is competition. This is just the way of the world.
Here is a longer view of the major OEMs plus the ODMs as a group since 2009:
Dell is the number one shipper of systems in the world, and the number one revenue generator, too, as it has been for quite some time now. Hewlett Packard Enterprise, including its H3C partnership in China, ranks second, as it has since Dell bypassed it. Dell is still growing, and HPE is flat, but it is not clear that either is actually profiting much by selling servers. (Someone has to build the machines, so we are grateful for the job they do and we hope that they can squeeze some more operating income out of their respective businesses.)
In the first quarter, Inspur (including its Power Systems partnership with IBM in China) and Lenovo and Cisco Systems were tied for the third place ranking in IDC’s numbers. Inspur brought in $1.22 billion in sales, up 36.4 percent and considerably faster than the revenue growth in the market at large, while Lenovo brought in $1.14 billion, up 3.9 percent, and Cisco brought in $1.05 billion, up 6.9 percent. The ODMs as a group brought in $4.55 billion in sales, down 1 percent, and the rest of the market (which includes IBM’s sales in this case) accounted for $4.34 billion, up 2.7 percent.
The alligator jaws that is the distribution of revenues by X86 and non-X86 server platforms continues to add teeth as 2019 gets started, and it is hard to remember the time, in the jaws of the Great Recession in fact, when sales of X86 and non-X86 systems were equal. Look at how quickly it diverged as the cloud builders and hyperscalers started building out their multi-million server node fleets.
Sales of IBM mainframe and Power platforms, the smattering of Arm systems, and the collection of other remnants of days gone past (mostly Sparc and Itanium systems) has reached a steady state of sorts of around $1.5 billion a quarter, give or take $200 million or so. The non-X86 iron is dominated by IBM System z mainframes with a healthy share of Power-based iron, and Arm machinery is still noise in the data. That could change, but the odds are increasingly less likely with a resurgent AMD. Arm chips may end up in networking and other gear in the datacenter, surrounding the servers even more than they do today, without making a dent in the actual servers – unless Intel or AMD stumble with their future X86 sales. After all the work that has been done with Arm server chips and the software stack, it would be interesting and fun to see Arm take hold. But for now, companies seem to be more focused on how much of their fleets might be based on Epycs as opposed to Xeons, and then there is always RISC-V to consider as another upstart eventually in the server space.
Whatever happens, it surely won’t be boring. That’s all we know.