If you look at the very long history of Cray, the company has done a remarkable job expanding its market, not just participating in it. And with the forthcoming “Shasta” systems and its “Slingshot” HPC-pumped Ethernet interconnect, which are due to start shipping at the end of this year, Cray will do it once again, setting itself up for a decade-long run and allowing it to push further down into the market than it has been able to do thus far.
At the start, the company that coined the term supercomputer was just selling exotic and elegant vector machines designed by Seymour Cray himself to a select handful of the largest high performance computing facilities on Earth. As supercomputing took off in the 1980s and became somewhat more mainstream, it was still dominated by the national labs and their parallel and smaller labs in academia and other government agencies, particularly those involved in security.
Back in November 2012, Cray paid $25 million to buy Appro International, an upstart supplier of clustered systems that lives on as the CS-Storm product line today, and further expanded its market across the national and academic labs, which often need capacity class clusters (which run many kinds of jobs simultaneously) instead of the capability class machines like the current XC systems from Cray (which tend to run fewer and bigger jobs that must scale across the hardware). The addition of Appro has been a boon for Cray. And not just because it gave the company two different product lines to sell and two different revenue streams that run on their own cycles, but also because it gave Cray two different ways of thinking about supercomputing. As it turns out, both will be embodied in the forthcoming Shasta machines – as we discussed back in October when Cray previewed the Shasta machines and Slingshot interconnect – allowing Cray to consolidate its product lines in a way that better reflects how companies would like to do high performance computing and still allowing it to tailor the racks, compute, storage, and networks as necessary to meet performance, capacity, and price points.
Having dealt with the up and down cycle of the supercomputer space, Cray has been pushing aggressively into commercial accounts that also need supercomputers, albeit often on a smaller scale. The company’s long term goal is to get about a third of its revenues from commercial companies, and based on what Cray’s top brass said this week in talking about its financial results for the fourth quarter of 2018, despite the contraction in the supercomputing market, Cray has been able boost its commercial sales to counterbalance it somewhat.
“Overall, our revenue from commercial customers grew nicely for the year, more than doubling in revenue and finishing the year at over 20 percent of our total revenues – our highest percentage ever,” Pete Ungaro, Cray’s chief executive officer, said on the call with Wall Street analysts. “Commercial growth is a big part of expanding our TAM, and we are very pleased with the progress we have made here.”
Commercial customers generally do not like to talk about what they are doing when it comes to HPC because they think of this as strategic, competitive advantage. This is in stark contrast to big government and academic HPC centers, which like to brag about their machines and the important work they are doing, not the least of which because many of these systems are wholly or partially paid for with public funding, one way or another. But Ungaro could nonetheless hint at what deals Cray did in the commercial sector during the fourth quarter.
A “major energy company” installed a big XC50 supercomputer and ClusterStor arrays for its parallel file system, which Ungaro said was one of the largest commercial installations it had ever done. In this case, the energy company is using the system for simulation and modeling of energy deposits, and is also working with Cray to integrate machine learning techniques with its simulations and has also installed a Urika analytics appliance to that end. (It is unclear if it is a Urika XC, based on Cray’s “Aries” interconnect like the XC50 system mentioned above, or a Urika-CS, which is based on the CS-Storm systems and either InfiniBand interconnect from Mellanox Technologies or Omni-Path interconnect from Intel.) Beyond this, Cray installed clusters at an existing aerospace customers in the United States and at another large commercial customer – industry unspecified – in the Asia/Pacific region.
Any time revenue share is more than doubling, as Cray’s sales commercial entities did in 2018, that is a good thing, and without this CS-Storm business, which we think has been a major driver of sales in commercial accounts, Cray would have very likely declined. (Those comments obviously take out the effect that building and delivering the original “Aurora” pre-exascale supercomputer at Argonne National Laboratory, which could have potentially been accepted in 2018 and which was worth $200 million. With delays in 10 nanometer manufacturing and other issues with its “Knights Mill” parallel processor, Argonne pulled the plug on that original Aurora machine in September 2017, and redid the deal with Intel and Cray to get the first exascale system, the Aurora A21, slated for 2021.)
For the year, which is the best way to look at Cray since the HPC sector is notoriously lumpy quarter to quarter, Cray grew sales by 16.2 percent to $455.9 million, a little bit better than the $450 million that Wall Street expected, and if you do some guesswork and peel out the commercial business, then just about all of that growth between 2017 and 2018 can be attributed to Cray’s success here. This is a solid footing on which to push Shasta systems when they become available in late 2019 and ramp in 2020 and beyond.
Given that the Shasta systems will have a wide variety of processors, multiple form factors that include direct liquid cooling and air cooling, and an HPC-style interconnect that is based on Ethernet (as well as support for InfiniBand and Omni-Path for those who still want it), Cray will be able to field systems that look and feel, from the viewpoint of systems software and networking, more like the Ethernet clusters that are commonly deployed in enterprises, we expect for Shasta to have even better traction in commercial sites than the XC and CS series machines did. It doesn’t hurt that these Shasta machines will be deployable for either HPC simulation and modeling or machine learning training and inference workloads, and will also bring storage onto the same Slingshot network to boost performance and manageability.
As for the core supercomputing market, Ungaro reminded Wall Street that the bidding activity in 2016 and 2017 fell by 60 percent compared to bumper crop 2015 year that Cray had. The company’s backlog has been steadily growing as supercomputing is recovering, and bookings stood at more than $600 million in the final quarter of 2018 was turned in.
“The market has begun to rebound with customers starting procurements for new systems and an overall improved velocity of deals as opportunities move through the sales process at a more normalized rate,” Ungaro explained, referring to this core supercomputing sector. “While momentum in the market is improving, a significant portion of the opportunities continue to be for delivery in 2020 and beyond. As a result, while we continue to expect 2019 revenue growth to be modest, it has a potential to be another strong year for bookings.”
A lot hinges on how the Shasta systems and the Slingshot interconnect are received by the market. Ungaro said that the Slingshot ASICs are back from the fab and prototypes are being tested now. Shipments will begin at the end of 2019, but as is the case in the supercomputing space, revenue for machines can’t be booked until the machines are accepted by customers and that means revenue will ramp through 2020 and beyond. Cray has bid its Shasta systems for the US Department of Energy’s CORAL-2 program, which will bring exascale systems to Oak Ridge National Laboratory and Lawrence Livermore National Laboratory, with a third exascale system above and beyond the “Aurora” A21 system that Intel and Cray are already building as an option for Argonne National Laboratory. Add it all up, with machines costing somewhere between $400 million and $600 million a pop, and one of these deals could swing the revenues of any of the players. We presume that IBM, Hewlett Packard Enterprise, and Intel are also independently bidding on the CORAL-2 procurement alongside Cray.
With just one of these CORAL-2 systems plus its normal run-rate of sales, Cray could turn in the best year in its history, and that is saying something considering how well the company did in 2015. All Ungaro could say is that Cray submitted proposals in May of last year and cannot comment any further.
In the meantime, HPC center and commercial customers who have to buy systems now will do so, and those who can wait for Shasta and who want to try out the new Cray architecture will wait it out. Cray is nonetheless expecting modest growth this year, will be spending money doing the final development and testing for Shasta and Slingshot as 2019 rolls by, and this will add to its costs. But Cray is investing for the long haul and expects to get about a decade out of the Shasta architecture. As far as we are concerned, this is exactly the attitude and strategy the company should be taking.