A Deep Dive Inside The HP Cloudline Servers

If you can’t beat them, join them. That is the old adage that compelled server juggernaut Hewlett-Packard to strike a partnership with original design manufacturer Foxconn last year for a new line of minimalist servers that HP could sell against other ODM upstarts that have stormed the datacenters of hyperscale and cloud service providers in recent years. That phrase also covers HP’s decision to join the Open Compute Project and embrace the vanity free server movement – at least for a select group of customers up in the stratosphere of scale.

HP did not have a choice but to find a manufacturing partner like Foxconn. The combination of the supply chains and engineering expertise gives the Cloudline machines leverage over Dell, Lenovo, Cisco Systems, and other tier one server makers who also want to sell into the same hyperscale and cloud markets. Many expect these customers could represent half of server shipments in the next several years. A lot of people have been taking a stab at how much iron this part of the market will represent. Here is a chart that Antonio Neri, general manager of the Enterprise Group at HP, showed at the Open Compute Summit 2015 conference in San Jose, where the Cloudline machines were launched:

HP is looking at money, not shipments in the chart above, and is putting together revenues from servers sold to run hyperscale, cloud, big data, and HPC workloads. This represented about 40 percent of the $40 billion or so in server revenues in 2013. But by 2017, HP’s forecasts show that these segments of the market will account for around 50 percent of a slightly larger market. This is too big of a market to walk away from, obviously. But HP has an IBM PC-System x problem, and Foxconn is helping solve that. With HP splitting itself into an enterprise business that sells servers, storage, switches, software, and services and another business that sells PCs and printers, it is about to split its supply chain leverage for CPUs, memory, disks, and other components in half. The minute that IBM got out of the PC business, it lost a lot of its pricing leverage in the component market (notably with Intel), and it was only a matter of time before IBM would not be making enough money in X86 servers to justify the effort. Lenovo, on the other hand, has a growing PC and client supply chain, and adding the System x business gives it more leverage. HP needed a partner not only for low-cost manufacturing, but to preserve its pricing in the supply chain.

With the launch of the Cloudline of machines developed in conjunction with Foxconn, HP will, for the first time in decades, bifurcate its X86 server product line. Way back in the early days of the X86 server business, when the commercial Internet did not yet exist, Compaq had a ProSignia line for small and medium businesses and a ProLiant line for midrange and enterprise customers. But after a while, everyone wanted all of the extra goodies to make a more reliable server, so HP just sold ProLiants. In a way, the Cloudline minimalist machines are a return to the bare-bones ProSignia line, albeit for machinery aimed at customers who have tens of thousands of machines humming along in massive datacenters, not a single box tucked under a desk somewhere in an office.

HP and Foxconn have been partners in one form or another for 30 years, and the two companies were pretty vague about how this Cloudline partnership would work when the deal was announced last April. For the Cloudline machines, HP and Foxconn worked together on creating a set of stripped-down machines using the most common processors, BIOS, memory, disks, and power supplies. Foxconn makes the motherboards and does the metal bending, while HP puts its brand on the machines and wraps services, support, documentation, and testing around them. Those latter points are a big deal for the large enterprises that want minimalist machines but who are used to a lot more hand holding from vendors than they are going to get from an ODM.

We were originally given the impression by HP that the advent of the Cloudline systems and last year’s Apollo 6000 and Apollo 8000 machines pretty much marked the end of the ProLiant SL4500 and SL6500 “scalable systems” machines from HP, which offered some of the densest plain CPU and hybrid CPU/GPU computing on the market. But HP has backed away from this notion and John Gromala, senior director of hyperscale management within HP’s Enterprise Group, explained to The Next Platform that the Cloudlines are not meant to replace any ProLiant machines, but to complement them. That said, it still seems reasonable that customers who were buying ProLiant SL machines will now go in one of two directions, as HP said originally. If they are HPC shops, they will gravitate towards the air-cooled Apollo 6000 and water-cooled Apollo 8000 systems. Hyperscale, cloud, and service provider customers looking for clusters of cheap machines to run web, caching, analytics, parallel data store, and warm and cold storage workloads will adopt the Cloudline iron, where they might have gone with ProLiant SL machines in the past. And as we said, we expect that more than a few customers will try to build HPC clusters using the Cloudline machines, too. It is not clear how accelerators will be integrated into the Cloudline systems, but presumably this will be done.

The Cloudline machines also represent HP’s formal launch into the Open Compute ecosystem, something that was inevitable if HP didn’t want to lose out its substantial hyperscale server and storage business to ODMs. Microsoft joined the Open Compute Project last year and released its server designs into the wild, and that means any Open Compute partner can make and certify their machines to that specification and bid on Microsoft’s enormous infrastructure contracts. The HP partnership with Foxconn gives Microsoft a large and reliable source for its server and storage needs behind the Azure cloud, the Bing search engine, the Office 365 online suite, and the Xbox gaming network. These machines might have been made by Foxconn or Quanta or any number of other ODMs had HP not joined up with Foxconn, and now it can protect its portion of the deal. Presumably Dell, which has been Microsoft’s other big supplier, is also building OCP-compliant machines on behalf of the software and cloud giant.

It also stands to reason that HP will be using the Cloudline machines to build out its own Helion public cloud. There is no way that HP could afford to use ProLiants in its own cloud, not considering the cut-throat pricing in the cloud market these days.

The Feeds And Speeds

The Cloudline machines all use Intel’s “Haswell” Xeon E5-2600 v3 processors and are certified to support the future “Broadwell” Xeon E5 v4 chips when Intel delivers them, presumably this fall. The Cloudlines are all two-socket systems, and they come in three different types of form factors: Rack machines that slide into regular 19-inch racks, another set of Open Compute rack machines that are compatible with the 21.5-inch Open Rack used by Facebook in several of its datacenters or the 19-inch Open Cloud Server chassis used by Microsoft. The third type of machine, which HP and Foxconn did not say much about at the event, will be a modular system that fits in a 19-inch rack that puts four server nodes in a 2U enclosure. The latter is a popular configuration among hyperscalers and cloud builders these days, but as we have pointed out before, most of these customers deploy rack, not modular, systems.

There are three models in the “bare iron” Cloudline rack lineup and all three use Intel’s C610 chipset. The CL1100 is the most elemental of the machines and is aimed at basic web infrastructure workloads and its “ultra-optimized” for low cost and basic performance. This CL1100 comes in a 1U enclosure and supports Xeon E5 chips that burn up to 120 watts and the system has only eight memory slots supporting DDR4 memory sticks with 4 GB or 8 GB capacities. The system has one PCI-Express x16 slot and supports the OCP mezzanine card slot, which can be used to add network adapters to the machine. The system comes with two 1 Gb/sec Ethernet ports and another two 10 Gb/sec ports can be added. The machine has a single 400 watt power supply and two 3.5-inch disk bays; the power supplies and disks are not hot pluggable.

The Cloudline CL2100 is also a 1U machine, but it has more storage – either four 3.5-inch drives or eight 2.5-inch drives – and has sixteen memory slots instead of the eight in the CL1100. The CL2100 has a 10 Gb/sec Ethernet port and two 1 Gb/sec ports, and a 650 watt power supply. The system has three PCI-Express 3.0 slots: One x16 slot, one x8 slot for an OCP-compliant 10 Gb/sec adapter card, and one x8 slot for a SAS storage mezzanine adapter card. HP says that this CL2100 machine is aimed at companies building infrastructure clouds and is also suitable for web caching and search applications.

The CL2200 is a 2U machine that has the same sixteen memory slots plus more room for disk storage and has more peripheral slots as well. HP says that this box is suitable for Hadoop, cloud databases, and NoSQL data stores. The machine has three storage and mezzanine slots in addition to one x16 and two x8 expansion slots for various peripherals. The CL2200 also has four 1 Gb/sec Ethernet ports, with the optional 10 Gb/sec port available through the OCP mezzanine card. The machine can have a dozen 3.5-inch disks in the front plus two 2.5-inch disks in the back; all of them are hot pluggable.

HP has not announced pricing on these Cloudline machines, but Gromala tells The Next Platform that these minimalist CL rack machines will offer prices that are about 10 percent cheaper on a base configuration compared to regular ProLiant machines in bare bones configurations, and as much as 25 percent less costly on heftier setups. These base Cloudline rack servers will be manufactured in Foxconn’s facilities in China, and as volume ramps, it will be able to do manufacturing in other regions.

The Cloudline rack machines will have all the IMPI, KVM, and SNMP management interfaces that companies expect as well as a set of Linux toolkits that will allow for BIOS configuration and replication of settings across nodes. The machines are certified to run Red Hat Enterprise Linux 7.0 and CentOS 6.6, which seem to be the preferred operating systems. But SUSE Linux Enterprise Server 12, Canonical Ubuntu Server 14.04, and Microsoft Windows Server 2012 R2 are all supported on the machines, too.

Targeting Facebook, Microsoft And OCP Friends

On the Open Compute front, customers who want to buy Open Compute machines like those used by Facebook and Microsoft, HP and Foxconn have two different machines they are bringing to market. Gromala says that HP will be starting with the machine that is compatible with the 21.5-inch Open Rack first – that’s the Cloudline CL7100 – with proof of concept customers getting machines in the third quarter and volume shipments in the fourth quarter. HP and Foxconn are also working together on a chassis and sled that adheres to Microsoft’s Open Cloud Server design. The delivery schedule of this latter machine, called the Cloudline CL7300, was not disclosed.

The details on these two Open Compute machines were a little thin, but here is what we know. The CL7100 fits into an Open Compute sled and puts two Xeon E5-2600 v3 processors on the sled; chips with 120 watt or lower thermal envelopes are supported, and the future “Broadwell” Xeon E5 chips will also slot into these nodes. The server has sixteen DDR4 memory slots and the fastest 2.13 GHz memory can be plugged in. The third-width node has room for six 2.5-inch drives, which plug into the front of the tray, and the system board supports the OCP mezzanine card that allows for 10 Gb/sec and 40 Gb/sec adapter options.

Here’s the front view of the CL7100 in its stock picture:

And here is what the node looks like as seen from above out on the OCP expo floor:

You can see the cards for network connectivity on the left of the front of the chassis and the disk slots on the right. It is not clear what slides into the slot on the bottom right. The idea for the Open Rack node is that everything is front accessible in the node, particularly those elements that are hot pluggable; the node itself is hot pluggable, in a sense, inside of the Open Rack shelf. Nothing needs any special tool to be removed and replaced, which cuts down dramatically on service time. This is a convenience when a datacenter has a few hundred machines, but it is an absolute necessity when a datacenter has 100,000 machines and somewhere around 1 to 2 percent of them are failing at any moment.

The Cloudline CL7300 is the system that not only fits the Microsoft Open Cloud Server design, but which has been acquired by Microsoft for its datacenters. Here’s a picture of the CL7300 sled being held by Tim McNamara, who is senior director of technical program management for cloud computing at Microsoft:

The CL7300 is based on a two socket motherboard, with eight memory slots per socket like most of the other Cloudline machines. The sled has room for four 3.5-inch drives and a carrier card in the center that snaps in four 2.5-inch disks or SSDs. The Microsoft-designed motherboard also has four M.2 flash modules, which can hold up to 1 TB of flash memory these days, and as McNamara told The Next Platform, it probably won’t be long before flash is dense enough that the M.2 cards will be sufficient flash that the precious space used by the carrier can be used for some other peripheral. The Microsoft Open Cloud Server chassis can accept 24 of these half width server nodes in a 12U chassis; the chassis has six 1,200 watt power supplies to feed all of the nodes. Each node has one PCI-Express 3.0 x16 slot and one mezzanine card that can have 10 Gb/sec or 40 Gb/sec ports coming off the node and passing through the back of the chassis to networks.

There are no obvious pricing comparisons to make between the HP ProLiant line and these two Open Compute Cloudline servers. But given how lean and mean they are and how much less hyperscale machines cost compared to traditional enterprise servers, the savings should be substantial.

Gromala tells The Next Platform that HP has no plans at this time to add 64-bit ARM server chips to the Cloudline family, but that is something that could be done fairly easily if one of the ARM chips takes off in the hyperscale arena. There is no minimum engagement for a customer to buy the Cloudline machines, but Gromala says that the line is aimed at customers who consume at least thousands of servers per year, not dozens. Odds are that HP and Foxconn will take any order that comes in, though.