• When Will AI Replace Traditional Supercomputing Simulations?

    May 15, 2017

    The science fiction of a generation ago predicted a future in which humans were replaced by the reasoning might of a supercomputer. But in an unexpected twist of events, it appears the it is the supercomputer’s main output—scientific simulations—that could be replaced by an even higher order of intelligence.

    While we will always need supercomputing hardware, the vast field of scientific computing, or high performance computing, could also be in the crosshairs for disruptive change, altering the future prospects for scientific code developers, but opening new doors in more energy-efficient, finer-grained scientific discovery. With code that can write itself based …

    Read more

  • The Next Battleground for Deep Learning Performance

    April 28, 2017

    The frameworks are in place, the hardware infrastructure is robust, but what has been keeping machine learning performance at bay has far less to do with the system-level capabilities and more to do with intense model optimization.

    While it might not be the sexy story that generates the unending wave of headlines around deep learning, hyperparameter tuning is a big barrier when it comes to new leaps in deep learning performance. In more traditional machine learning, there are plenty of open sources tools for this, but where it is needed most is in deep learning—an area that does appear to …

    Read more

  • Escher Erases Batching Lines for Efficient FPGA Deep Learning

    April 25, 2017

    Aside from the massive parallelism available in modern FPGAs, there are other two other key reasons why reconfigurable hardware is finding a fit in neural network processing in both training and inference.

    First is the energy efficiency of these devices relative to performance, and second is the flexibility of an architecture that can be recast to the framework at hand. In the past we’ve described how FPGAs can fit over GPUs as well as custom ASICs in some cases, and what the future might hold for novel architectures based on reconfigurable hardware for these workloads. But there is still …

    Read more

  • Taking the Heavy Lifting Out of TensorFlow at Extreme Scale

    April 25, 2017

    There is no real middle ground when it comes to TensorFlow use cases. Most implementations take place either in a single node or at the drastic Google-scale, with few scalability stories in between.

    This is starting to change, however, as more users find an increasing array of open source tools based on MPI and other approaches to hop to multi-GPU scalability for training, but it still not simple to scale Google’s own framework across larger machines. Code modifications get hairy beyond single node and for the MPI uninitiated, there is a steep curve to scalable deep learning.

    Although high performance …

    Read more

  • A Look at Facebook’s Interactive Neural Network Visualization System

    April 12, 2017

    There has been much discussion about the “black box” problem of neural networks. Sophisticated models can perform well on predictive workloads, but when it comes to backtracking how the system came to its end result, there is no clear way to understand what went right or wrong—or how the model turned on itself to arrive a conclusion.

    For old-school machine learning models, this was not quite the problem it is now with non-linear, hidden data structures and countless parameters. For researchers deploying neural networks for scientific applications, this lack of reproducibility from the black box presents validation hurdles, but for …

    Read more

  • Neuromorphic, Quantum, Supercomputing Mesh for Deep Learning

    March 29, 2017

    It is difficult to shed a tear for Moore’s Law when there are so many interesting architectural distractions on the systems horizon.

    While the steady tick-tock of the tried and true is still audible, the last two years have ushered a fresh wave of new architectures targeting deep learning and other specialized workloads, as well as a bevy of forthcoming hybrids with FPGAs, zippier GPUs, and swiftly emerging open architectures. None of this has been lost on system architects at the bleeding edge, where the rush is on to build systems that can efficiently chew through ever-growing datasets with …

    Read more

  • Scaling Deep Learning on an 18,000 GPU Supercomputer

    March 28, 2017

    It is one thing to scale a neural network on a single GPU or even a single system with four or eight GPUs. But it is another thing entirely to push it across thousands of nodes. Most centers doing deep learning have relatively small GPU clusters for training and certainly nothing on the order of the Titan supercomputer at Oak Ridge National Laboratory.

    The emphasis on machine learning scalability has often been focused on node counts in the past for single-model runs. This is useful for some applications, but as neural networks become more integrated into existing workflows, including those …

    Read more

  • Rapid GPU Evolution at Chinese Web Giant Tencent

    March 24, 2017

    Like other major hyperscale web companies, China’s Tencent, which operates a massive network of ad, social, business, and media platforms, is increasingly reliant on two trends to keep pace.

    The first is not surprising—efficient, scalable cloud computing to serve internal and user demand. The second is more recent and includes a wide breadth of deep learning applications, including the company’s own internally developed Mariana platform, which powers many user-facing services.

    When the company introduced its deep learning platform back in 2014 (at a time when companies like Baidu, Google, and others were expanding their GPU counts for speech and …

    Read more

  • Can FPGAs Beat GPUs in Accelerating Next-Generation Deep Learning?

    March 21, 2017 Linda Barney

    Continued exponential growth of digital data of images, videos, and speech from sources such as social media and the internet-of-things is driving the need for analytics to make that data understandable and actionable.

    Data analytics often rely on machine learning (ML) algorithms. Among ML algorithms, deep convolutional neural networks (DNNs) offer state-of-the-art accuracies for important image classification tasks and are becoming widely adopted.

    At the recent International Symposium on Field Programmable Gate Arrays (ISFPGA), Dr. Eriko Nurvitadhi from Intel Accelerator Architecture Lab (AAL), presented research on Can FPGAs beat GPUs in Accelerating Next-Generation Deep Neural Networks. Their research …

    Read more

  • An Early Look at Startup Graphcore’s Deep Learning Chip

    March 9, 2017

    As a thought exercise, let’s consider neural networks as massive graphs and begin considering the CPU as a passive slave to some higher order processor—one that can sling itself across multiple points on an ever-expanding network of connections feeding into itself, training, inferencing, and splitting off into multiple models on the same architecture.

    Plenty of technical naysay can happen in this concept, of course, and only a slice of it has to do with algorithmic complexity. For one, memory bandwidth is pushed to limit even on specialized devices like GPUs and FPGAs—at least for a neural net problem. And second, …

    Read more

Previous Articles

Mellanox

The Next Platform Weekly

Tap the stack to painlessly subscribe for a weekly email from The Next Platform, featuring highlights, analysis, and stories from the week directly from us to your inbox with nothing in between.