The people in the systems business at Hewlett-Packard have finally caught the DevOps bug. As the company prepares to split itself into two, with HP Enterprise focusing on the datacenter and looking for a way to grow revenues and profits, the company has come up with the idea of “composable infrastructure” as a means of distinguishing its hardware from that of its competitors while also, HP hopes, giving it an edge over the competition from Dell, Lenovo, and a slew of original design manufacturers who have not, as yet, created tools for programmatically configuring and managing servers, storage, and the fabrics that link them.
One of the constant themes here at The Next Platform is that interesting new approaches to designing or managing infrastructure start up on high in the hyperscale or HPC arenas and are gradually adapted and adopted at the high-end of the enterprise and cloud spaces. Such is the case with what HP is calling composable hardware, which is by no means a new idea. Facebook has a tool called Kobold that it created to automagically provision servers in its datacenter and another tool called FBAR, short for Facebook Automated Remediation, that fixes crashed machines that together do the work that would require several hundred system administrators in each of the company’s datacenters. Facebook talked publicly about these tools back in 2011, so they are probably many years older than this. Microsoft has been operating its homegrown Autopilot tool to manage the configuration of its servers and storage as well as doing automatic remediation of issues on the iron since 2004. Google’s Borg cluster management software, which shuttles work around the search engine giant’s massive infrastructure, shares many of the same features (conceptually speaking) as Autopilot, according to a paper published in April. (You can see our discussion of Google’s Borg and related Omega management tools here.) Amazon Web Services no doubt has sophisticated management tools that allow for its iron to be programmatically animated and bossed around, but it has not talked publicly about them.
HP is no slouch when it comes to systems management, thanks in large part to its acquisition of Compaq back in 2001. Compaq’s Insight Control server provisioning and migration tools were a key differentiator for the company as it peddled iron into the dot-com boom, and were a jewel in the Industry Standard Servers business at HP for many years. HP’s own OpenView network monitoring tools were ubiquitous back in the day, too.
Program Your Hardware Like A Hyperscaler
While HP still sells Insight Control and OpenView, these are human tools with point and click interfaces, and that is not how hyperscalers do things – and not how enterprises increasingly want to do things. To that end, in the fall of 2013 HP updated its server management stack with its OneView follow-on to Insight Control, which has some REST APIs exposed to make server management programmable, much as the server provisioning and management tools are at the hyperscalers. The OneView tool was developed in conjunction with PayPal, eBay, Intuit, Bank of America, JPMorgan Chase, and dozens of other companies, and it automates the provisioning of HP ProLiant machinery, and in its initial release could only monitor up to 640 machines after they had been provisioned. While this scale has been boosted slightly, HP has set its sights on a much larger problem with Project Synergy, which will see HP bring tools to market along with partners that will make servers, storage, and switches all equally programmable for provisioning and management.
Paul Miller, vice president of marketing for converged datacenter infrastructure at HP, tells The Next Platform that HP is not ready to divulge its full roadmaps for Project Synergy, but does say that the effort is a natural next step after HP has done lots of work to converge the hardware that underpins applications.
“With convergence, it was about putting servers, storage, and networks together under a single management framework to make it easier for IT to stand up systems,” explains Miller. “We are still going to invest in that, but we see a need for companies to make their infrastructure more agile and to enable them to move to a continuous delivery model not just for applications, but for the systems underneath them. That’s what composability is all about. We have aggregated units of compute, storage, and networking, but the units have become kind of static. We want to go one step further and decompose those elements and then recompose them to align them specifically to applications.”
The IT market already has many elements of the needed toolset for malleable application and system configuration available, including Chef, Puppet, and Ansible on bare metal, Docker for software containers (even if it is not yet fully mature), and VMware ESXi/vSphere and OpenStack for hypervisor virtualization. HP is not going to recreate the wheel on these tools, and in fact it probably could not get customers to adopt an alternative should it have had the gumption to attempt to create one to any of these tools. So what HP is sensibly doing is positioning OneView as a hub that all of these tools a little higher up the stack above the iron will plug into, and then promising customers that it will expand OneView to cover more servers, storage, and switching gear it goes further down the Project Synergy roadmap.
At this point, HP is opening up and exposing the APIs in the OneView tool so others can rally around it and hook their various wares into it. The HP Labs, HP Cloud, HP Enterprise, and HP Software units of the company are all kicking in resources for the Project Synergy effort, and HP will be soon releasing a software development kit so partners in the composable infrastructure effort can hook into OneView smoothly. HP is working right now to link into the OpenStack cloud controller and its Ironic bare metal provisioning API, and the idea here is to “make it so you don’t need a staff of PhDs to make it work,” as Miller put it. Other orchestration layers will eventually be brought into the fold, too, says Miller, without being specific.
Over the long haul, taking perhaps years, HP wants to end up with system management tools that deliver something akin to what Facebook, Google, Microsoft, and Amazon have, which is a flexible pool of physical, virtual, and container resources for compute and file, block, and object storage for storage that can be diced and sliced on demand as applications expand and contract on the infrastructure.
The Project Synergy effort will take years for HP to flesh out with partnerships, the good news is that most enterprises that are HP server, storage, and switching customers have not yet made the jump to continuous application delivery and the DevOps model that breaks down the wall between programmers and administrators to enable this. So in this regard, HP is not too far behind the wave of change in the datacenter. That said, other companies are launching products that emulate the job scheduling capabilities of the hyperscalers – notably Mesosphere, with its open source Mesos tool that is inspired by Google’s Borg and that now has Kubernetes container management added on – and it would not be impossible (although difficult) for them to add server, storage, and network provisioning underneath their schedulers. (The support matrix across servers, switches, and storage can quickly get out of hand for provisioning tools, which is why hyperscalers have only two or three core platforms.)
If HP wants to get traction with OneView, it may have to open source it, or it may have to go out and acquire Chef, Puppet, or Ansible so it can create a single toolset analogous to Insight Control. Perhaps HP has come to the conclusion that it is too late for such a move, or perhaps it is waiting until after HP Enterprise breaks off from the PC and printing company to do acquisitions to turn Project Synergy into a product with a reasonable revenue stream a few years hence. The odds are that HP will not spend a lot of money on the effort and make Project Synergy a way to very tightly control its specific iron and make it malleable; the odds do not favor HP expanding out to competitive iron, unless customers start demanding it.