AI At The Edge Is Different From AI In The Datacenter

Today’s pace of business requires companies to find faster ways to serve customers, gather actionable insights, increase operational efficiency, and reduce costs. Edge-to-cloud solutions running AI workloads at the edge help address this demand. Placing compute power at the network edge, close to the data creation point, makes a vital difference for near-real-time use cases. Handling functions like computing algorithms and data processing on local devices rather than moving those workloads to the cloud data center dramatically reduces latency.

In manufacturing, locally run AI models can rapidly interpret data from sensors and cameras to perform vital tasks. For example, automakers scan their assembly lines using computer vision to identify potential defects in a vehicle before it leaves the plant. In a use case like this, very low latency and always-on requirements make data movement throughout an extensive network impractical. Even small amounts of lag can impede quality assurance processes. On the other hand, low-power devices are ill-equipped to handle beefy AI workloads, such as training the models that computer vision systems rely on. Therefore, a holistic, edge-to-cloud approach combines the best of both worlds. Backend cloud instances provide the scalability and processing power for complex AI workloads, and front-end devices put data and analysis physically close together to minimize latency.

For these reasons, cloud solutions, including those from Amazon, Google, and Microsoft, play a vital role. Flexible and performant instances with purpose-built CPUs, like the Intel Xeon processor family with built-in AI acceleration features, can tackle the heavy lifting for tasks like model creation. From there, prepared models transfer to lightweight edge-based systems that collect and interpret data.

Low Power Devices At The Edge

Device manufacturers like Arduino design ruggedized, low-power edge devices. Many cost less than $100, and Arduino users can combine a few or thousands of devices running machine learning models. Arduino complements its devices with the Arduino Cloud, supported by performant and scalable Amazon EC2 instances. Arduino also uses the Intel Distribution of OpenVINO toolkit that provides developers with pre-published models to make the development process for cloud-to-edge solutions less labor-intensive.

An agriculture business, for example, used Arduino’s solution to maximize crop yields. The solution involves sensors that feed the edge devices data like soil moisture and wind conditions to determine the amount of water necessary for the healthiest crops. The technology helps farmers avoid overwatering and reduces the costs of running electric water pumps.

In another use case, a manufacturer dependent on its precision lathes used sensors in combination with Arduino’s devices to detect anomalies like minute vibrations that cloud signal an impending equipment problem. It’s much more cost-effective for a business to plan scheduled maintenance than to encounter an unexpected failure that halts production.

Recruiting Exceptional Athletes

AiScout, an application developed by UK-based ai.io, provides another excellent edge-to-cloud example. The company’s AiScout application bridges the gap between amateur athletes hoping to be discovered by talent scouts and recruiters needing exceptional team players. Historically, recruiters traveled extensively to evaluate athletes’ skills, making the process expensive and time-consuming. Players faced a different problem. Even those with exceptional skills could go unnoticed by a talent scout if they live in other countries or remote areas.

Any athlete can use the free aiScout app on their edge device – in this case, a phone with a camera – to record themselves, showcase their technical skills, and upload the video. On the backend, cloud instances accommodate and showcase millions of athletes’ videos, including those with sub-optimal lighting or unintended camera movement. ai.io’s platform for motion capture can then analyze videos, gather performance data, and develop 3D visualizations to aid recruiters. Talent scouts can then peruse videos from anywhere, on their own devices, and find the athletes who can best supplement their existing team’s roster.

After an extensive evaluation to find an ideal cloud solution to support its application, ai.io chose a combination of Amazon EC2 instances with Intel Xeon processors and Amazon DL1 instances with Intel Gaudi accelerators rather than GPUs for model training. According to ai.io, its choice of varied Intel-based instances increased its inference performance by 50 percent to 200 percent while lowering costs by 40 percent compared to GPU-based solutions. Today, the wide adoption of the edge-to-cloud aiScout solution benefits players and recruiters. Talent scouts who often spent 18 months evaluating and signing players can complete the same process in only two weeks.

Enabling Digital Pathology

In the healthcare field, edge-to-cloud solutions also serve vital functions. Traditionally, highly skilled doctors examined patients’ tissue biopsies, X-rays, CT scans, and more for possible health-related concerns. However, no human is perfect. Overwhelmed radiologists and pathologists pouring over hundreds of images per day could inadvertently miss a tiny detail that matters. By applying AI to the mix, doctors have an extra set of eyes to evaluate scans rapidly and in exceptional detail. If AI identifies any anomaly, medical professionals can focus additional time and attention on the image to determine whether it warrants further medical intervention. AI can help pathologists achieve better patient outcomes without disrupting their workflow.

An edge-to-cloud solution proves ideal for this medical use case, allowing clinicians to share images quickly and securely with others within a hospital network using a 5G network.

Since each type of disease indicator requires a different AI model, resources like Intel Smart Edge help manage multiple trained AI models among a system of hospitals. The HepatoAI platform also benefits from the Intel OpenVINO toolkit and the OpenVINO model server, which assist in AI processing and inferencing across multiple medical applications.

The solution allows pathologists to upload image data securely from edge-based workstations to a centralized private cloud. That backend server solution relies on Intel Xeon Scalable processors for fast throughput and built-in AI acceleration. Together, these technologies accommodate the gigantic data volumes associated with medical images so other in-network doctors can access that information from their workstations using a secure browser. For more details about this solution, check out the article.

Getting Started With Edge-To-Cloud Solutions

As the use cases in this article illustrate, AI supported by edge-to-cloud capabilities can provide immense value to organizations of all sizes. The end-to-end approach combines the physical and virtual worlds to help companies improve upon legacy processes, build new revenue streams, get near real-time insights, collaborate better, reduce costs, and so much more.

Businesses evaluating or preparing to implement holistic AI solutions face many considerations, but a few suggestions might help ease the process.

Some companies prefer a more holistic and turnkey solution, while others build a custom solution from scratch. Sometimes, a combination of the two approaches provides the most significant benefit. Successful implementations typically start by considering the desired business outcomes and working backward. Evaluating a proven, edge-native software platform is a terrific starting point for whichever approach an organization selects.

Embracing solutions based on open standards will help integrate heterogeneous components and avoid lock-in with a single vendor. An open approach can also help future-proof cloud and edge investments. If one component requires an upgrade, it remains compatible with other solution components. It’s helpful to choose proven technologies supported by many ecosystem partners with ready-made libraries and tools to implement AI rapidly. In addition to other technologies described in this article that facilitate AI across distributed environments, components like the Intel vPro platform powered 13th Gen Intel Core processors can deliver a potent combination of speed and security. For visualization workloads, Intel also offers a converged edge media platform combining the open Intel Datacenter GPU Flex Series, Intel Xeon CPUs, and fast ethernet connections to bring the components together.

Cloud providers like Amazon, Microsoft, and Google offer customers immense help when implementing edge-to-cloud AI solutions. They bring the expertise and robust solutions – plus the security, speed, and scale – to help customers cost-effectively embrace the power of AI.

At its Vision Conference in April, Intel announced several products to extend its edge platform and silicon roadmap. This year, Intel seeks to advance AI PCs with its latest Intel Core Ultra client processors (code-named “Lunar Lake”), enabling more AI processing at the edge. For backend edge infrastructure, forthcoming “Granite Rapids” Xeon 6 Processors will feature several optimizations for GenAI.

Companies and scientific institutions may choose different approaches to their edge-to-cloud AI solutions. The planning and implementation process may take time, but the long-term benefits make the journey worthwhile.