What’s Ahead for Supercomputing’s Balanced Benchmark
March 21, 2018
We all know about the Top 500 supercomputing benchmark, which measures raw floating point performance. But over the several years there has been talk that this no longer represents real-world application performance.
This has opened the door for a new benchmark to come to the fore, in this case the high performance conjugate gradients benchmark, or HPCG, benchmark.
Here to talk about this on today’s episode of “The Interview” with The Next Platform is one of the creators of HPCG, Sandia National Lab’s Dr. Michael Heroux. Interestingly, Heroux co-developed HPCG with one of the founders of the Top 500 LINPACK benchmark, prompting many early on to suggest HPCG was going to compete with the Top500 as the go-to benchmark for the HPC community.
As it turns out, HPCG and LINPACK are bookends—each measuring different aspects of supercomputing system performance. This focus on balance and meaning for real-world workloads has prompted the benchmark to grow in the number of submissions, beginning in 2014 with just a few to 115 now, including many of the top-performing systems on the Top 500 list.
In our conversation with Heroux, we touch on the evolution of the benchmark in terms of development and adoption, how the results highlight differences between machines (some like the #1 system in China perform very well on LINPACK but not quite as well on HPCG, for instance), and what the addition of more numerous and diverse and accelerated machines means for the future of the list and results.
Heroux says the vendor response has been positive and that even though Top 500 has notoriety, the early development efforts included vendors from the beginning to ensure their support. While this is a more demanding benchmark for HPC sites to run compared to LINPACK, especially on machines like Trinity with its mix of Knights Landing and Haswell nodes that need to be split to run the benchmark effectively, Heroux says centers are generally open to doing the extra legwork in part because these same efforts mirror what they need to do with their own applications.
In addition to talking about GPUs on HPCG, we also talked about where Arm processors stand for running HPCG off-air. Heroux says that they are hard at work with the Arm team to beef up the needed math libraries but we should start seeing results on Arm in the near future.
HPCG benchmark details and most recent results can be found here