You have no doubt heard the one about making it up in volume, a jokey phrase that people use when a business has to crank out more and more widgets to stay in the same place. In the systems business, this is no joke. It is, in fact, the way of business. But the situation is more complex – and not as dire – as it might look. We think companies are getting incredibly better value out of their systems.
The latest figures for server shipments and revenues, covering the third quarter that ended in September, are out from Gartner, and while both key metrics of the business were up sharply, the longer trends in the business show that, as a whole, companies are not radically increasing their spending on systems. There are tremendous pressures that are keeping the once lucrative revenue streams from servers down, even as vendors sell more and more X86 iron into datacenters that want more for their money every year.
Jeffrey Hewitt, research vice president for servers at Gartner, tells The Next Platform that there are a few factors at work here, and these are familiar to those of us who watch the high-end datacenter market. In the enterprise segment of the market, server consolidation continues to push companies to buy beefier machines, and they can cram more virtual machines onto each new generation of systems, which keeps downward pressure on shipments. A machine with 50 percent more cores and 50 percent more memory does not cost 50 percent more – or better still, 100 percent more as it might otherwise in years gone by – and that is in turn. At cloud builders, the same economics are at play as these companies deploy beefier machines and carve them up for their customers.
The stagnation in server revenues is partly a reflection of an architectural shift away from big iron NUMA machines towards clusters for new workloads that might otherwise have run on such systems. Hyperscale and HPC organizations very rarely use such big iron, although the latter does sometimes employ NUMA systems as well as proprietary interconnects that offer lower latency and more concurrency than commodity clusters. This helps push up revenues and average selling prices, to a certain extent.
We also think that the shift from storage array appliances to more generic storage servers is also buoying the server market, and a similar effect will occur with network function virtualization.
Further militating against this downward pressure on both revenues and shipments are the rapidly expanding workloads of the consumer-facing hyperscale companies like Google, Microsoft, Baidu, and Facebook, who buy their systems in blocks of tens of thousands and have millions of machines in their fleets in many cases. These companies buy minimalist machines, often from original design manufacturers in Asia, and also buy their own components, driving the price of the system way down and therefore the revenue stream. Hewitt believes that those companies that design their own systems and contract out for manufacturing, as Google, Microsoft, Amazon, Facebook, and such do, is approaching 20 percent of the market as far as Gartner can tell, with a fair amount of semi-custom machinery being sold by Supermicro, Dell, Hewlett Packard Enterprise, and others that boosts this number even further.
“The volumes keep going up because of hyperscale deployments and because of enterprises that are operating at a webscale and are starting to mimic them,” says Hewitt. Goldman Sachs and Fidelity Investments are two examples, both of which are deploying Open Compute designs, and many others are starting to adopt minimalist machines that may or may not adhere to Open Compute standards, such as the Cloudline machines that HPE makes in conjunction with Foxconn or the new Dell DSS systems.
The plot lines for server revenues and shipments since the beginning of the Great Recession are illustrative. Take a look:
The Great Recession technically started in December 2007, but it took almost a year to become evident and that is when, as you can see, server revenues and shipments took a dive.
Very large Unix and mainframe systems, which take a long time to get approval and are a bit like a freight train in terms of trying to stop them, continued on their paths to the datacenter during the early part of the recession and X86 servers, where companies could prioritize new workloads and look for efficiencies through virtualization, took a more immediate hit. The market bottomed out in 2009 and started to recover. But as the revenue trend line shows, excepting the occasional fourth quarter spike (more often than not driven by hyperscalers), system sales have been more or less flat even as shipments have continued to trend upwards.
If you do the math, the revenue from the first three months of 2007, before the Great Recession started, came to $39.5 billion, by Gartner’s reckoning, and for the first three months of 2015, revenues were $40.5 billion. That is a very modest 2.6 percent growth compared to eight years ago – which is not a lot of growth. However, server shipments have increase by a factor of 10X over that period, rising 26 percent from the 6.42 million machines shipped in the first three quarters of 2007 to the 8.12 million machines shipped in the first three quarters of 2015. Considering that average utilization has gone up – let’s say it is on the order of 25 percent now, compared to 10 percent back then – and the amount of computing in the box has risen – probably by a factor of 10X or so – the amount of effective computing that companies are getting for the dollar has risen dramatically.
If you multiply all of those factors out, that comes to a factor of 32X more compute capacity in those two three-month periods. So while revenues are essentially flat, and shipments are growing linearly, capacity might be growing linearly with a much steeper curve or it might even be exponential. (We will try to actually map this out at some point.) The issue is this: If you take the money into account, then the price/performance that organizations are getting from machines in 2015 is about a factor of 48X higher than in 2007. Having said all of that, even with average utilization at 25 percent, a tremendous amount of value is going up the chimney as heat in the datacenter.
Still, this is progress that the simple revenue and shipment chart above cannot show. None of this means that the server market has not become an extremely tough place to make money, unless you are Intel, Microsoft, Red Hat and a few component suppliers doing high-end networking or flash storage. This is cut-throat, make no mistake about it. But the end users are benefitting – right up to the point where the supply chain can’t make a living any more. Then, Intel may have to take over for real.
Digging Into The Third Quarter
In the three month period ending in September, Gartner figures that companies consumed 2.76 million servers, a 9.2 percent increase from the third quarter of 2014. Revenues rose by 7.5 percent to $13.5 billion. Average selling prices fluctuate across all architectures depending on how many hyperscale deals get done and where IBM is in its System z mainframe cycle, but the general trend is for average selling prices to decline, down from $6,053 in the third quarter of 2007 to $4,887 eight years later in Q3 2015.
In the quarter, X86 machinery accounted for 99.1 percent of shipments at 2.74 million machines and 85.3 percent of revenues, driving $11.52 billion in sales. HPE sold 609,746 machines to get its $3.82 billion in sales and Dell pushed $501,262 boxes to get $2.42 billion. Having taken over the System x business from IBM, Lenovo more than tripled shipments to 242,005 and brought in $1.07 billion, a factor of 6.5X increase in revenues. Cisco Systems shipped 86,581 machines in the quarter, up 5 percent, but these machines drove $885.6 million in sales, up 13 percent. Cisco’s period of hypergrowth is over, although it is trying to get into a new market with its UCS M Series hyperscale machines. Chinese system maker Huawei Technology significantly outgrew the market, with $462 million in sales, up 71.7 percent, against 134,163 shipments, up 43.6 percent. Inspur, another up-and-coming Chinese system maker with global aspirations in enterprise and HPC, made the top five shipper cut, with 99,333 machines sold during the quarter, up 8.9 percent, more or less in pace with the market, and had $396 million in sales, up 12.8 percent.
Unix systems based on RISC or Itanium architectures saw an 11.5 percent revenue slide to $1.06 billion, and all of the Unix system vendors saw declines. IBM was down 7.5 percent to $613.5 million, Oracle was down 13.9 percent to $232.7 million, HPE was down 28.2 percent to $146.2 million, and Fujitsu was down 27.7 percent to $35.1 million. All told, RISC/Itanium system shipments rose by a tiny 1.1 percent to 23,262 units in the third quarter of 2015.
We would say that the rumor that Fujitsu might be considering an ARM processor instead of its own Sparc64 RISC chips for future supercomputers could turn out to be economically inevitable. And by the way, ARM servers do not even make it as more than a rounding error in the Gartner data, but Hewitt is watching this space carefully like the rest of us.
Add it all up, and HPE is the top of the revenue stack, with $3.68 billion in sales, followed by Dell with $2.42 billion, and IBM at number three with $1.33 billion. Lenovo ranked fourth and Cisco ranked fifth. All others comprised $4.12 billion, and saw 11.6 percent revenue growth, half as fast again as the overall market. There are lots of interesting things going on in that Others category.