Orchestrating HPC Engineering in the Cloud
February 2, 2017 Jeffrey Burt
Public clouds have proven useful to a growing number of organizations looking for ways to run their high-performance computing applications to scale without having to limit themselves to whatever computing capabilities they have in-house or to spending a lot of money to build up their infrastructure to meeting their growing needs.
The big three – Amazon Web Services, Microsoft Azure and Google Cloud – have rolled out a broad array of compute, networking and storage technologies that companies can leverage when their HPC workloads scale to the point that they can no longer be run on their in-house workstations or they have HPC applications to run that their own systems don’t have the capacity to handle. The public cloud providers offer a variety of systems powered by different processor architectures – x86, Power and ARM, for example – accelerators like FPGAs and GPUs, networking options from Ethernet to InfiniBand, and storage like SSDs and HDDs.
As we’ve talked about previously, a stumbling point has been less the technology and more the licensing of HPC software, which has not kept up with the trend towards the on-demand world of the cloud. However, a couple of years ago, ANSYS, which makes engineering and simulation software, poked a hole in that problem when it moved its code into AWS, adopting a cloud licensing model for its HPC applications. Through its ANSYS Enterprise Cloud, the software vendor opened up the pathway for customers to begin migrating some of their workloads to the cloud, whether they needed a way to quickly get more compute capacity when their needs exceeded their own infrastructure capabilities or they don’t have the budget or desire to build up their datacenters to handle the HPC workloads.
Through its Enterprise Cloud, ANSYS gave customers greater agility and scalability in running their workloads, and a flexible licensing model to make it easier for them to embrace the cloud architecture. Company officials today took another step in that direction in announcing a partnership with Cycle Computing, a cloud orchestration and management vendor that has made its bones in the HPC field. Cycle’s CycleCloud software platform provides the middle ground between the end user and the public cloud that is aimed at making moving high-end simulation workloads to the cloud more palatable for HPC organizations, from ensuring the availability and security of compute resources to the management of those workloads in the cloud.
By partnering with Cycle, ANSYS is hoping to give engineers more confidence in leveraging the cloud to their advantage. Despite the benefits public clouds can offer, the engineering field is still in the “early adoption phase,” Judd Kaiser, program manager for cloud computing at the company, tells The Next Platform. Engineers by nature tend to be a conservative bunch, and while there are advantages to cloud computing, there also is a comfort to having the hardware bought and paid for and sitting inside the walls of the building. “They like the idea of scaling up and down based on their needs, but they’re still accustomed to” buying systems and running them in-house, Kaiser said. They also have questions around utilization and cost.
Still, they’re coming around and looking deeper into what they can do with the scalability and flexibility of the cloud. Engineers are understanding that with the cloud, their workloads no longer have to sit in queues awaiting available compute resources, and that their ability to scale isn’t limited by the amount of capacity in their datacenters. The combination of the ANSYS Enterprise Cloud and CycleCloud is aimed at ensuring that the resources they need are available and used for only the amount of time needed. The Cycle software also gives engineers better access to AWS Spot instance usage, which can further drive down the costs of using the cloud.
“Customers have the need to scale large projects or the amount of simulations they’re doing is outgrowing their datacenters or they’re going into HPC and don’t want to spend the money and time to build a new data center,” he said.
Cycle Computing over the years has made its bones by being a cloud orchestration and management platform vendor for the HPC space, collecting a growing number of high-end customers in such industries as pharmaceuticals, financial services, insurance, manufacturing and government agencies, such as the federal Food and Drug Administration. These companies have often been concerned about the cloud over such issues as security and performance, but they’re increasingly finding the ability to rapidly scale up such workloads for such jobs as drug discovery and financial simulation a significant advantage. And scale they can, with Cycle enabling them to run workloads on cloud clusters with tens of thousands of cores. For example, ran a 70,000-core cluster in AWS for HGST to enable the hard drive maker to model new helium drives in high resolution. The vendor also has worked with the Broad Institute on a cancer research effort that took advantage of 50,000 cores on the Google Cloud.
Cycle CEO Jason Stowe, speaking with The Next Platform, ticked off a list of benefits that the cloud can bring, from accelerating workloads and enabling massive scalability without limitations to eliminating operational and hardware concerns as well as the zero-sum game within companies concerning access to resources. No longer does it come down to a one employee getting access to compute capacity at the expense of someone else’s workload. Engineers, through the ANSYS Enterprise Cloud, are increasingly more willing to move workloads into the cloud, understanding that the more simulations they can run, the faster they will be able to bring products to market and the more competitive they will be, Stowe said.
Organizations won’t want to run all their applications in the cloud, but they’ll want to be able to move data and workloads between their in-house datacenters and cloud environments, and will want to be able to manage multiple applications in multiple clouds. Cycle’s platform is designed to enable all that.