A New Era of Open Competition for HPC Begins

Every year people ask me what I think the biggest news story is at the annual Supercomputing Conference (SC). I look at product announcements, of course, and SC15 promises to have some big ones, but ultimately, the biggest stories are the ones that have context and meaning beyond individual products, standards that are emerging or mergers that are evolving. In that light, this year’s most important development may have occurred already.

On Thursday, the Linux Foundation – the titular stewards of open-source advancement – announced a new collaborative effort to promote standards and innovation in high performance computing: OpenHPC. The logo salad on the openhpc.community website is comprised of dozens of inaugural participants, including the two most prominent HPC server vendors (Dell and HP), a rebounding supercomputing titan (Cray), the dominant processor provider (Intel), several notable HPC software providers (Allinea, Altair, ANSYS, MSC Software), and an international who’s-who list of trailblazing research labs. To the casual observer, it might look like the entire community pulling together in support of a common, open goal. Except that it’s not.

Richard Stallman (aka “rms”), the open-source zealot who was the progenitor of GNU “copyleft” agreement, notoriously clarified the “gratis versus libre” distinction between freeware and free software by quipping, “Think of free as in free speech, not as in free beer.” In the case of OpenHPC, there is a similar thesaurus trick at play. In this case it may be more appropriate to think about open competition than about open access.

Although all participants in OpenHPC are democratically listed on equal footing, the clear instigator is Intel. And regardless of its laudable merits – and there are many – OpenHPC is motivated to a significant degree by industry reaction to last year’s big story, OpenPOWER.

Since its introduction in 2013, the OpenPOWER Foundation has gathered its own stable of members. As the name implies, this consortium is driven primarily by IBM, though it gets substantial amplification from Mellanox and NVIDIA, two other companies that view Intel as a natural competitor. Collectively, OpenPOWER seeks to challenge Intel’s hegemony in what has been a homogeneous market. A recent Intersect360 Research report revealed that over 90% of HPC systems shipped since the beginning of 2014 utilized Intel processors.

 

And yet, the apparent near-totality of Intel’s umbra belies a potential vulnerability. Further research studies show that accelerated computing has reached a tipping point for HPC, with over half of all systems expected to move to accelerated elements in the next year or two. It is here where Intel’s dominance cannot be assured. NVIDIA had a substantial early-mover advantage over Intel with CUDA and its GPU computing strategy. At present, 34 of the top 50 applications for HPC offer GPU support (including two pending release). Intel has Xeon Phi in the market, with the Knights Landing upgrade coming soon, packaged both as an accelerator/coprocessor and as a native standalone processor, but the software ecosystem around Xeon Phi has catching up to do.

Similarly, Intel is coming from behind in the interconnect market. The OmniPath architecture promises to be one of the most important product announcements in the coming year, as Intel moves to integrate network communication with computation. But Mellanox dominates the InfiniBand space currently, with newly announced smart switches and co-design architectures, making a strongly competitive case.

Open versus Open

If OpenHPC vs. OpenPOWER boils down to Intel-and-Friends vs. IBM-and-Friends, it’s worthwhile to examine who’s on which side.

• Server vendors: HP, Dell, Cray, Lenovo, Sugon, and Fujitsu are among the notable names in OpenHPC (Fujitsu is particularly interesting given its previous work with SPARC), while OpenPOWER has IBM (of course), as well as Hitachi, Penguin, and Tyan. And playing both sides, Atos/Bull, NEC, and Inspur are members of both. (Penguin, mentioned in Linux Foundation’s press release but not on the OpenHPC web site, may be in both as well.) SGI is the most notable HPC system company that doesn’t appear in either.
• Processor vendors: The alliances go beyond Xeon and Xeon Phi vs. POWER and NVIDIA. With its pending acquisition of Altera, Intel is picking up a leading FPGA vendor that has had a Platinum Sponsor role in OpenPOWER. OpenPOWER has responded by moving Xilinx up to the top sponsor line, joining Micron at the Platinum level. AMD is in neither.
• Storage vendors: Ignoring the fact that server vendors like HP, Dell, and Cray also sell storage, OpenHPC doesn’t have any storage representation, a seeming lack, given the importance of I/O and data movement. Given IBM’s data-centric vision and strategic investment in Spectrum storage, one might expect OpenPOWER to have the harder time attracting storage members, but DDN and HGST are both members. Neither Seagate nor Panasas is in either, nor are the major enterprise storage vendors, EMC, NetApp, and HDS.
• Linux: SUSE in OpenHPC. Ubuntu in OpenPOWER. RedHat in neither.
• Software vendors: Given the need to optimize applications to evolving architecture frameworks, this is a seemingly important category, and OpenHPC shows leadership here, with Allinea, Altair, ANSYS, and MSC Software all listed as members. OpenPOWER has none so notable.
• Supercomputing centers: Where it can, the HPC user community will try to test all options before committing, and indeed, several major sites are members of both consortia, including Jülich, Oak Ridge, Lawrence Livermore, and TACC.
• Hyperscale: One of the great wildcards in how the HPC market will develop is the influence of hyperscale players. Google is the biggest of them all and a Platinum level sponsor of OpenPOWER.

Whither ARM?

If there’s a key technology area not addressed by either consortium, it’s ARM. A few notable supercomputing sites – including Barcelona Supercomputing Center, a member of OpenHPC – have been testing ARM architecture for its potential applicability to HPC, and while its usage is still in a nascent trial stage, it still represents another potential paradigmatic shift in the next few years.

It seems unlikely that ARM technologies will find a seat at the OpenHPC table, though it bears emphasis that both consortia are “open” to any member. OpenPOWER could be a different story, as the computational elements therein are already diversified. Furthermore, such a lightweight processor could be viewed as complementary to IBM’s data-centric strategy. It remains to be seen whether members of the ARM ecosystem will join OpenPOWER or OpenHPC – or start their own team (which may be justifiable solely based on the puns enabled by welcoming members into OpenARMs).

And speaking of openness, there is one critical difference in how the prime players have implemented the four-letter adjective. When IBM founded the OpenPOWER consortium, it underscored the non-closed nature by adopting ARM’s model, allowing the OpenPOWER technology to be licensed. Intel is unlikely to open this door; in fact, Intel doesn’t like anyone to peek through the mail slot.

Open Choice

Against this shifting architectural landscape, the HPC industry is continuing its migration away from standardization, back toward specialization. End users will choose which platforms to optimize their applications for. Not everyone will make the same choice, and once a particular workload starts down any one path, the inertia of investment will make it difficult to change tacks.

OpenHPC is an important announcement – perhaps the most important we’ll see at SC15 – as it begins to coalesce the ecosystem that will surround Intel architecture for the next several years. Each OpenHPC member will have its role in optimizing technologies for high performance, and in this endeavor, Intel needs its partners as much as its partners need Intel. This symbiosis is ultimately the goal of OpenHPC, to demonstrate that no one company controls the fate of the industry, and by extension, the advancement of scientific research and innovation. That’s an idea we can all open our minds to.