HP Opens Up Homegrown Network Operating System

The open networking movement just got a little bit more open. While a number of network operating systems already exist that are based on the Linux kernel and that include a number of open source add-ons that provide the functionality of closed, proprietary network operating systems, thus far there has not been one that is fully open source and that also has commercial support.

Today, that changes as Hewlett-Packard, which has a fairly large datacenter switch business, launches OpenSwitch, a new Linux-based and open source network operating system that is inspired by its own Comware and ProVision stacks for switches that have their heritage in 3Com and HP devices, respectively.

OpenSwitch is not just a network operating system, Mark Carroll, chief technology officer for the HP Networking division, tells The Next Platform. The idea is to make OpenSwitch a community of developers, user companies, and vendors that are seeking to create and foster an open networking stack akin to that which we have on servers with the Linux operating system. The founding members, aside from HP, are switch makers Accton Technology and Arista Networks, switch ASIC makers Broadcom and Intel, and virtual networking software provider VMware.

This is an interesting mix of partners. Arista has its own Linux-based Extensible Operating System (EOS) for its switches, and has not committed to using OpenSwitch. But Arista is happy to have any organization disrupting the hegemony of Cisco Systems’ iOS and NX-OS operating systems in the datacenter switches of the world. Intel is supportive of any and all open networking operating systems because, like Arista (one of its customers for certain switches), it wants to see a mix of ASICs and X86 processors in intelligent and standardized networking devices (much as has happened in servers over the past twenty years) and dislodge Cisco. Breaking the NOS from the hardware suits its purposes and may even give Intel a chance to sell its silicon against Broadcom, Cavium Networks, Mellanox Technologies, and others.

Broadcom, with 65 percent share of datacenter switch ASICs, is already in the position that Intel craves in networking and has to embrace both proprietary and open NOS software if it hopes to maintain its lead. Accton is one of the ODMs that is benefitting from the open networking wave, which is trying to break the switch free from the NOS that makes it do stuff, much as operating systems are no longer tied to servers. In fact, Accton is HP’s hardware partner for its Alto line of whitebox switches, announced back in February running the Cumulus Linux NOS from Cumulus Networks and the PicOS NOS from Pica8. It is not clear what VMware gets from this, except that it can take its NSX virtual switching and now have physical switching alongside of it.

“We have done a lot of work over the past 18 months to build a full-fledged, modular, modern Linux-based operating system, but we are putting it out there and we are not going to have exclusive rights to it.”

While HP has nothing against Cumulus Linux or Pica8, the company does have networking expertise of its own and it wants to cultivate a completely open source and full-featured NOS that it has influence over, even if it cannot outright control it. (This is the right way to do open source.) The idea is not so much to become the Red Hat of network operating systems – although that might be a nice thing – but rather to bring everyone together to create an carrier-grade product backed by a broad community of common interest and enterprise-class tech support and implementation services.

The OpenSwitch NOS is based on a raw Linux kernel that is streamlined and hardened for networking, and that has the popular Quagga Layer 3 routing protocol software added to it. The Layer 2 switching functionality, which Carroll says is a bear for anyone to implement properly, is shiny new code that is inspired by its own features in the Comware and ProVision NOS software that runs on its closed switches today. Similarly, the VXLAN network virtualization part of the OpenSwitch stack, which allows for Layer 2 networks to be stretched over Layer 3 links to make it all look like one big, flat network for server virtualization hypervisors, is inspired by HP’s proprietary code, as are link aggregation (LAG) and numerous other protocols required in the modern datacenter.

“We have done a lot of work over the past 18 months to build a full-fledged, modular, modern Linux-based operating system, but we are putting it out there and we are not going to have exclusive rights to it,” Carroll explains, saying that the OpenSwitch community will have a board that guides its development. “It is a flexible foundation for developers in the open switch community to create high-quality network enhancements specifically for the datacenter environment for themselves specifically or to be provided as a service.”

Carroll says that HP looked at Comware and ProVision and decided that it needed to start with a fresh Linux base because of the scalability issues that switching has to span these days and for the large number of access methods and analytics and telemetry that modern network operations requires. And that is a centralized, subscription-publish model akin to all kinds of controllers that have automated various aspects of the datacenter in recent years.

“Every daemon and every application has to go through this, and it allows for scale not only across Layer 2 and Layer 3, but also for analytics and for a subscription-publish model that allows changes to take effect immediately once they are made in the network. We believe this is the method that is necessary to get scale and performance in the network, and that is why we did it from scratch.”

The OpenSwitch NOS is based on the raw Linux kernel, and has a mix of C++, Go, and Java programs running on top of it and databases for managing the control plane and data plane as well as JSON interfaces for the API stack. It is not clear what kernel HP has chosen, but Debian seems to be popular in the network community.

Debian is the kernel of choice for Open Network Linux, an open NOS that was proposed back in November 2013 to the Open Compute Project, started by Facebook to open source datacenters and gear. Big Switch Networks is behind the Open Network Linux effort, which was formally accepted into the OCP back in March of this year, and its Switch Light stack of networking software is based on it. But the full Switch Light stack is not open source. Neither is Cumulus Linux, by the way, but both Switch Light and Cumulus Linux do have licenses and commercial-grade support if you want to run them on bare metal switches. While NetVisor from Pluribus Networks is based on open source technologies, the same holds – the full NOS needs a license and support contract and you can’t get the source code. Ditto for PicOS from Pica8. It is based on Debian Linux but the source code for the switching, routing, OpenFlow, Open vSwitch, and other functions are not out there in the wild for anyone to grab and change. (Incidentally, it is not so easy to grab all the bits that comprise Red Hat Enterprise Linux and make your own distro these days, either.)

“OpenSwitch is completely open, and you can create your own distro or your own business with this,” says Carroll. And it is likely that some may try to do just that, and others will grab this as the foundation for their NOS stack if they see it take off and try to move up the value chain. It all depends on how good the OpenSwitch implementation is and how it evolves.

At the moment, HP is initially targeting OpenSwitch to run on its Alto line of switches, which use Broadcom’s “Trident-II” and “Tomahawk” ASICs. HP is not naming names, but Carroll says it will support other ASICs with OpenSwitch and will also backport it to its own custom ASICs used on its own line of datacenter switches. “Our diversity will be quite high, not only Broadcom,” says Carroll, referring to the OpenSwitch effort, not just HP’s product line. And when we asked about the prospects of OpenSwitch running on Cavium’s XPliant, Mellanox’ Spectrum, and Intel’s Fulcrum, Carroll said that he believed that “eventually OpenSwitch will run on all of them” because that is how open source works, but he was very careful to divulge any specific plans.

The OpenSwitch community will be located here, and the source code will be released soon. (Binaries are available now.) HP expects to have a commercial-grade version of OpenSwitch available by the second half of 2016. It will be interesting to see how these various open NOS projects work together – or decide not to.