Nvidia Pushes AI Out To The Edge, Where 5G Is Waiting

The image of Nvidia co-founder and CEO Jensen Huang pacing across the stage at a tech conference and talking about his company’s latest new markets for artificial intelligence, the latest partnerships with high-profile vendors, and looking into the near horizon for Nvidia’s next revenue stream has become a familiar one. Over the five that Huang has been telling audiences that AI, deep learning, deep neural networks, and the processing and analyzing of the skyrocketing amount of data being generated represented the future for Nvidia and its GPUs – and that none of it could happen as well without those GPUs – the company has rolled out a roster of products aimed at establishing its place in a rapidly changing IT world.

Nvidia is turning its attention to the edge and specifically to telecommunications companies, where the promise of 5G connectivity and software-defined 5G networks that will play significant roles in defining what the edge will become. In a keynote address this week the day before the kickoff of the Mobile World Congress LA 2019 show in Los Angeles, Huang again laid out his vision of the future at the edge and Nvidia’s role in it, from new and improved products to a range of partnerships that include Microsoft, Ericsson, and Red Hat.

There was announcement of well-known companies that have adopted Nvidia’s EGX Edge Supercomputing Platform, which was introduced in May at Computex as a software-defined offering that includes the EGX stack, complete with a Kubernetes plug-in, Nvidia container runtime, GPU monitoring tools and an Nvidia drives, all managed by the Nvidia GPU Operator, for automating lifecycle management of containerized applications with Kubernetes. Among the companies embracing EGX are BMW, Samsung Electronics, Proctor and Gamble, and Walmart.

 

At the same time, Huang spoke about the partnerships that will help drive the evolution of software-defined 5G networks and the introduction of Aerial, a software-development kit that will work with GPU-based off-the-shelf servers to drive the development of GPU-accelerated 5G radio access networks (RAN) and the services that will run on them. All of these announcements are aimed at an edge environment that will include huge numbers of sensors and other devices generating massive amounts of data that needs to be quickly processed and analyzed.

“With the convergence of AI, the Internet of Things and the approaching 5G infrastructure, the opportunity is ripe for companies to push their models beyond the data center to the edge, where billions of sensors are streaming data and making real-time decisions is a reality,” Adel El-Hallak, director of product management at Nvidia, wrote in a blog post. “Enterprises deploying AI workloads at scale are using a combination of on-premises data centers and the cloud, bringing the AI models to where the data is being collected. Deploying these workloads at the edge, say in a retail store or parking garage, can be very challenging if IT expertise is not available as one might have with data centers.”

As the edge evolves, businesses are looking to get more compute and storage capabilities closer to where the connected and intelligent connected systems and devices are located for faster access to the data and the insights that analytics and other tools can derive from it. AI increasingly is a key technology for the edge and 5G capabilities, which already is beginning to appear in some products and networks but promises to really ramp starting next year, will play a key role in the development at the edge, and Nvidia wants to play a significant role in all these areas.

“The first big bang of AI started in the cloud when big amounts of customer data started being gathered and they brought together big data and deep learning with Nvidia GPUs to build new AI models that could deliver a recommendation engine, chat box, and translation,” Justin Boitano, general manager of Nvidia’s enterprise and edge computing business, said during a call with journalists. “The next big wave of AI is now happening at the enterprise edge, to process data from billions of sensors that are being deployed in factories, stores and cities.”

The EGX platform is a way for companies to efficiently and quickly move AI out to the edge. The EGX stack comes with the system as well as a Linux distribution, containers and Kubernetes, and the GPU Operator technologies.

The platform – which includes Mellanox Technologies’ SmartNIC cards and Nvidia’s CUDA-X libraries – comes with software for a range of uses cases, such as the Aerial SDK for telcos that want to build virtualized 5G RANs, which will enable them to offer such services as smart factories, augmented and virtual reality and cloud gaming. There also is Metropolis, Nvidia’s software used in smart cities environments and intelligent video analytics applications.

BMW, Proctor and Gamble, and the other companies mentioned already are using EGX, Boitano said. The car maker is using it in its manufacturing site in South Carolina to analyze data from video cameras and sensors along the inspection line. P&G is developing AI-enabled applications for the platform to inspect products and packaging.

Walmart has set up an “array of cameras and sensors across 5,0000 square feet of their store, and they’re taking these IoT sensors’ data and processing it in real-time to understand what’s happening in the physical world, how to serve customers better,” Boitano said. “In order to process all the sensor data, they need to compute over 1.6 TB of data per second. Using AI on Nvidia GPUs to automatically alert associates to restock shelves, open up checkout lanes, to retrieve shopping carts, and then to ensure that the meat and produce departments remain fresh. The IoT evolution they’ll be talking about is really about sensors being put out into the world. Now AI is helping enterprises instantaneously process that data and respond in real time. AI is going to bring higher efficiency to factories, the stores and cities around the world.”

Nvidia’s push around 5G will come via products and partnerships. 5G promises significant improvements over 4G and LTE, including 1,000 times the bandwidth, a tenth of the lower latency and the ability to support millions of connected devices per square mile. It also comes with a feature called “network slicing,” which enables telcos to offer portions of the network for specific use cases, such as self-driving vehicles or a smart electrical grid.

“This enables telcos to support these resources and for them to deliver all the services that are becoming common in the datacenter and sharing that infrastructure across a wider set of cellular … towers,” Boitano said. “What they then want to do is elastically scale the right services based on consumer demand at any point in time in a way where everything is software-defined, even if they turn things on and off, depending on the demand on their network. However, until now, the most important workload for telcos couldn’t be easily software-defined, which is the 5G Radio Access Network.”

That challenge is what the partnership with Ericsson is looking to address, with the companies investigating ways to deliver a RAN solution that is virtualized and efficient in terms of size, power and cost. Nvidia and Red Hat – now owned by IBM – will develop ways to help telcos move to 5G networks that can run software-defined workloads and initially will 5G RANs to make AI-based workloads accessible at the edge. The companies, which announced a partnership earlier this year to push adoption of Kubernetes in datacenters, are looking to leverage Nvidia technologies like the EGX platform and the Aerial software and Red Hat’s OpenShift containerization software.

The partnership with Microsoft is aimed at more tightly integrating Azure and the EGX platform and drive more AI capabilities to the edge. Nvidia’s Metropolis framework for video analytics is now optimized to work with Microsoft’s Azure IoT Edge and Machine Learning offerings, and the companies announced a new, smaller form factor of the Azure Data Box Edge appliance powered by Nvidia’s T4 GPUs.

The companies also said that Nvidia-certified off-the-shelf servers that are optimized to run Azure IoT Edge and Machine Learning solutions are available from a range of OEMs, including Dell EMC, Hewlett Packard Enterprise, and Lenovo.