The Air Force Research Laboratory (AFRL) has officially inaugurated its four newest supercomputers in a ribbon cutting ceremony at the AFRL DoD Supercomputing Resource Center (DRSC). The new systems represent the agency’s first instance of being able to share HPC resources across multiple classifications.
AFRL DSRC Director Jeff Graham explained that that kind of multi-classification setup will avoid the extra expense of purchasing and deploying separate systems for each classification, as well as constructing special secure facilities for the machines. “The ability to share supercomputers at higher classification levels will allow programs to get their supercomputing work done quickly while maintaining necessary security,” Graham said. The multi-classification machines will be shared across the defense establishment, enabling the Department of Defense to save billions of dollars, according to Graham.
All four of the new supercomputers are HPE SGI 8600 systems, supplied by Hewlett Packard Enterprise. One of the them, known as Mustang, will be relegated to unclassified computing, while the three others, Voodoo, Shadow, and Spectre, will be devoted to classified work at the center, as well as across the DoD. Together, the systems will support research in hypersonic aircraft, in addition to other advanced work in air, naval, and ground weapon systems.
The four AFRL systems were part a larger deal between HPE and the DoD’s High Performance Computing Modernization Program (HPCMP) that included three additional HPE SGI 8600 systems for the Navy DSRC. All the machines are powered by 24-core Xeon Scalable Processors and use Intel’s Omni-Path fabric as the system interconnect. When it was originally announced in February 2018, the contract for the seven systems was valued at $57 million, which included five years of support.
Understandably, the Air Force is not releasing the particulars about its three classified supercomputers. However, it looks like the unclassified Mustang system is the largest of the bunch, performance-wise, claiming 4.9 petaflops of the 7.3 petaflops total of the four. Curiously, Mustang is not the speediest supercomputer at the AFRL center. That title goes to the 5.6-petaflop Thunder, a 2014-vintage SGI ICE X machine that is accelerated by Nvidia GPUs (Tesla K40M) and Intel Xeon Phi processors (7120P).
For the most part, Mustang is a straight-up CPU machine, powered by Xeon Platinum 8168 processors across 1,176 dual-socket nodes. The only exception to this is that 24 of those nodes are also equipped with a single Nvidia Tesla P100 GPU as a coprocessor. The Air Force didn’t specifically reference the use cases for the P100, but given the DoD’s overall interest in AI and machine learning, one can surmise that some of these GPU cycles will go toward such applications.
On the storage side, the four systems together will have access to a total 12 petabytes of Lustre storage, based on the DDN EXAScaler platform. Of this, Mustang has access to 8.4 petabytes, which seems like plenty for system of its size until you realize the older Thunder system can tap into 17.3 petabytes.
Deals like this point to the value of HPE’s acquisition of SGI in 2016 and how it’s made the company into the revenue leader in high performance computers. In this case, HPE is benefiting from the DoD High Performance Computing Modernization Program’s inclination to purchase HPC machinery with an SGI pedigree. Here is worth noting that two years after the acquisition, HPE still retains the SGI branding on these systems, which is rarely the case when a smaller OEM is subsumed by a larger one.
In any case, HPE certainly appears to have benefitted from the acquisition, and SGI’s former customers are probably a lot more comfortable that the technology is now under the umbrella of a much larger, more stable vendor. And since IBM is no longer a competitor in the x86 server space, thanks to it selling off the business to Lenovo in 2014, HPE has become the dominant OEM in high performance computing. In 2017, the company rang up $4.2 billion in HPC server sales, more than its next three rivals (Dell EMC, Lenovo, and Inspur), combined.
Defense applications aside, the SGI 8600, has been a popular system for a number of large HPC customers. The platform is the basis for Tokyo Tech’s 12.1-petaflop TSUBAME 3 supercomputer, NASA Ames’s 8.3-petaflop Electra system, and the National Renewable Energy Laboratory’s 4.9-petaflop Eagle machine. There are also a couple dozen petaflops of SGI ICE X deployments around the world, which would be natural candidates for the 8600 when those installations are upgraded.
Although the ribbon cutting ceremony took place this week, the Air Force said Mustang became operational in December 2018.