Performance per watt is garnering increasing attention these days with efficiency and sustainability less of a PR point of pride and far more of a TCO concern.
The benchmarking pros that comprise the SPEC performance evaluation organization recognize this and have gathered several member companies, including AMD, Nvidia, Ampere, IBM, and others for a new International Standards Group (ISG) around datacenter energy consumption.
The effort is centered around the SERT standard, which analyzes server efficiency across different application groups with various server-class processors and OS environments, both as standalone servers and clusters. While not new (the effort began in 2020) it is gaining steam with additional members and more a formal structure.
The Java-based benchmark suite appears relatively lightweight with completion time in under three hours, according to SPEC. Ease of reporting and reading results has been emphasized as well, with automatic submissions to certification programs and detailed reports for customers.
“The framework is designed to accommodate other program languages as well. A series of different workload characteristics are implemented to demonstrate the effectiveness of different server sizes and configurations,” according to SPEC.
“The workload also utilized SPEC’s concept of multiple load levels and details can be found in the design document. Regulatory programs are designed to foster continuous improvement, with thresholds for success rising as the industry progresses. The SERT suite is designed to match this paradigm by implementing a quick adoption process of new computing technologies.”
The cost for the latest SERT 2.0.6 is $2800. V2.0.6 adds supports for regulatory certification of Windows Server 2022 and includes the PTDaemon Interface v1.10.0.
In addition to the expected participating companies, government bodies are joining the ISG, including the U.S. Environmental Protection Agency (EPA) and China’s National Institute of Standardization, and trade groups, including Japan’s Ministry of Economy, Trade and Industry.
ISG is led by Klaus-Dieter Lange, a SPEC Director. Other members include Dell, Google, HPE, HP Inc., IBM, Inspur, Intel, Microsoft, and the University of Würzburg.
“High-quality, vendor-agnostic measurement tools, such as SPEC’s benchmarks, pair perfectly with SC39’s international standards for Sustainability, IT and datacenters to provide our planet with turn-key solutions vital to global energy conservation and the future of computing,” explains David Reiner, Chair of ISO/IEC JTC1 SC39.
“Together, ISG and SC39 will make it easier for countries to accelerate the adoption of critical standards, better measure their progress toward their energy and sustainability goals, and ease regulatory compliance through international standardization,” Reiner adds.
Real-World HPC Gets the Benchmark It Deserves
While nothing can beat the notoriety of the long-standing LINPACK benchmark, the metric by which supercomputer performance is gauged, there is ample room for a more practical measure. It might not garner the same mainstream headlines as the Top 500 list of the world’s largest systems, but a new benchmark …
AMD Rome Epyc Processor Shatters Numeric Benchmarks
Good news is continuing to gather around AMD’s second-generation “Rome” Epyc processors. The company’s newly minted Epyc 7H12 chip has established new highwater marks in four SPECrate benchmarks, as well as in High Performance Linpack (HPL). The results were reported by Atos, which ran the benchmarks on a dual-socket server …
One Deep Learning Benchmark to Rule Them All
Over the last few years we have detailed the explosion in new machine learning systems with the influx of novel architectures from deep learning chip startups to efforts from vendors and hyperscalers alike. With such processor diversity, standardizing performance metrics when there are so many variables is tricky, but it …
The utility of the SERT 2 score in comparison shopping for servers will hopefully become clearer with broadening adoption and reporting (like miles-per-gallon or liters-per-100km) as it currently looks like the result of a somewhat complicated (compound) calculation. I do think it will help foster some quite useful innovations from server designers though, as there is no real point in wasting a high-quality energy source (such as electricity) by generating waste heat, that then requires more elecricity to remove from the datacenter. In a related but slightly off-topic note, it seems that using low-quality energy sources (wood chips, coal, natural gas, etc…) to generate electricity, and then use that electricity to generate heat at home (a plan in California), seems rather more wasteful than using these sources directly to generate that heat in the home (without the double-conversion).