Time in the Sun Coming for Cloud FPGAs

Keeping an eye on how the largest cloud providers choose to invest in hardware is always interesting but it does not often shed much light on how emerging workloads are driving new investments. As an Amazon, Google, Microsoft, or other cloud, bringing the best in breed processor is simply a matter of course. For smaller cloud players that are focused on key workloads and user types, their investments in new hardware is perhaps a bit more telling.

Among those companies we watch that are building public clouds at smaller scale is Nimbix. Unlike the largest cloud companies, they are large enough to serve a diverse set of workloads but small (and open) enough to provide reasonable analyses of what people deploy and why. Further, they have been in front of a few key acceleration trends, including most recently, the addition of the Groq AI engine to their options.

As we described before, they were notable firsts to the cloud with a few other processors, most notably FPGAs. The company’s CEO, Steve Hebert, had FPGAs at the heart of the envisioned cloud he wanted to build in 2010. In November of that same year GPU availability in the Amazon cloud had just been announced—and at a time when they were most widely adopted in HPC, a community that had serious reservations about the cloud in general given latency concerns, among others. In short, the original idea of an FPGA acceleration cloud was quite forward-looking, perhaps a bit ahead of times. Hebert still thinks the vision has yet to be fully realized but thinks once it emerges they will find themselves well positioned to deliver a specialized FPGA cloud platform that pushes ahead of the rest.

Below is our interview at The Next FPGA Platform event in San Jose on January 22 where he describes potential for FPGAs and projected growth from a workload perspective.

https://youtu.be/n-l_-8EmkfI

When asked how workloads have changed for FPGAs over time he says bioinformatics was an important area for FPGA adoption, which has given way more recently to newer use cases in video and AI inference. He adds that standardization of hardware (Xilinx/Alveo) form factors has been a big deal for developers kicking the tires because they can take any application code and look at how it might be accelerated. While he contends that FPGAs in general (in our out of the cloud) are still nascent, the end of Moore’s Law, the ability to put machines in a cloud that can change to keep pace with new requirements, will keep pushing the demand for these devices.

Hebert says they are going to continue to invest in that initial vision with more devices. To speak to growth in FPGA use, from 2016-2019, the consumption of FPGAs within Nimbix grew 7X, he reveals. “I’m completely bullish on this. It’s inevitable. Reconfigurable computing is inevitable. We have the end of Moore’s Law ahead, which means we have to put machines in a cloud datacenter that can change.

In terms of where that growth is consumption of FPGAs is coming from, Hebert agrees that it is not that anticipated explosion in overall growth of FPGAs that’s driving things now. “It is still nascent. There are many developers, I’d liken it to the early days of CUDA in the hands of grad students on their laptops looking for any kernel that can be accelerated, that’s the mode we’re in now.”

In terms of what is still lacking for these users, Hebert says easing deployments and providing a simple on-ramp to start using the devices is at the top of list, which is where his teams are focused. “We launched Alveo in 2018, which was the first step in the next stage. We’ve worked with different hardware configurations but this was the first standard architecture that lets developers start solving challenging problems. There’s a lot of real estate on these devices.”

It’s refreshing to get some actual insight into what FPGA users are doing on various cloud platforms. Aside from a few use cases, Amazon’s been mostly mum on the topic. This is in stark contrast to the early days of GPU acceleration on its cloud. There were new case studies from Fortune 500 companies and the largest research centers almost weekly pushing how GPUs were being used and scaled in the cloud. It might be up to deeply invested players like Nimbix who are small enough to track adoption trends.