• Drilling Into Microsoft’s BrainWave Soft Deep Learning Chip

  August 24, 2017 Timothy Prickett Morgan

  

  There are a lot of different ways to skin the deep learning cat. But for hyperscalers and cloud providers who want to use a single platform internally as well as providing deep learning services to customers externally, they really want to have as few different architectures as possible in their datacenters to maximize efficiencies and to lower both capital and operational costs. This is particularly true when the hyperscaler is also a cloud provider.

  If Moore’s Law had not run out of gas – or at least shifted to lower octane fuel – then the choice would have been easy. …

  Read more

• An Early Look at Baidu’s Custom AI and Analytics Processor

  August 22, 2017 Nicole Hemsoth

  

  In the U.S. it is easy to focus on our native hyperscale companies (Google, Amazon, Facebook, etc.) and how they design and deploy infrastructure at scale.

  But as our regular readers understand well, the equivalent to Google in China, Baidu, has been at the bleeding edge with chips, systems, and software to feed its own cloud-delivered and research operations.

  We’ve written much over the last few years about the company’s emphasis on streamlining deep learning processing, most notably with GPUs, but Baidu has a new processor up its sleeve called the XPU. For now, the device has just been demonstrated …

  Read more

• IBM Highlights PowerAI, OpenPower System Scalability

  August 9, 2017 Nicole Hemsoth

  

  The golden grail of deep learning has two handles. On the one hand, developing and scaling systems that can train ever-growing model sizes is one concern. And on the other side, cutting down inference latencies while preserving accuracy of trained models is another issue.

  Being able to do both on the same system represents its own host of challenges, but for one group at IBM Research, focusing on the compute-intensive training element will have a performance and efficiency trickle-down effect that speed the entire deep learning workflow—from training to inference. This work, which is being led at the T.J. Watson …

  Read more

• Google Research Pushing Neural Networks Out of the Datacenter

  August 8, 2017 Nicole Hemsoth

  

  Google has been at the bleeding edge of AI hardware development with the arrival of its TPU and other system-scale modifications to make large-scale neural network processing efficient and fast.

  But just as these developments come to fruition, advances in trimmed-down deep learning could move many more machine learning training and inference operations out of the datacenter and into your palm.

  Although it might be natural to think the reason that neural networks cannot be processed on devices like smartphones is because of limited CPU power, the real challenge lies in the vastness of the model sizes and hardware memory …

  Read more

• Wrenching Efficiency Out of Custom Deep Learning Accelerators

  August 7, 2017 Nicole Hemsoth

  

  Custom accelerators for neural network training have garnered plenty of attention in the last couple of years, but without significant software footwork, many are still difficult to program and could leave efficiencies on the table. This can be addressed through various model optimizations, but as some argue, the efficiency and utilization gaps can also be addressed with a tailored compiler.

  Eugenio Culurciello, an electrical engineer at Purdue University, argues that getting full computational efficiency out of custom deep learning accelerators is difficult. This prompted his team at Purdue to build an FPGA based accelerator that could be agnostic to CNN …

  Read more

• Technology Requirements for Deep and Machine Learning

  July 14, 2017 Rob Farber

  

  Having been at the forefront of machine learning since the 1980s when I was a staff scientist in the Theoretical Division at Los Alamos performing basic research on machine learning (and later applying it in many areas including co-founding a machine-learning  based drug discovery company), I was lucky enough to participate in the creation and subsequently to observe first-hand the process by which the field of machine-learning grew to become a ‘bandwagon’ that eventually imploded due to misconceptions about the technology and what it could accomplish.

  Fueled by across-the-board technology advances including algorithmic developments, machine learning has again become a …

  Read more

• When Will AI Replace Traditional Supercomputing Simulations?

  May 15, 2017 Nicole Hemsoth

  

  The science fiction of a generation ago predicted a future in which humans were replaced by the reasoning might of a supercomputer. But in an unexpected twist of events, it appears the it is the supercomputer’s main output—scientific simulations—that could be replaced by an even higher order of intelligence.

  While we will always need supercomputing hardware, the vast field of scientific computing, or high performance computing, could also be in the crosshairs for disruptive change, altering the future prospects for scientific code developers, but opening new doors in more energy-efficient, finer-grained scientific discovery. With code that can write itself based …

  Read more

• The Next Battleground for Deep Learning Performance

  April 28, 2017 Nicole Hemsoth

  

  The frameworks are in place, the hardware infrastructure is robust, but what has been keeping machine learning performance at bay has far less to do with the system-level capabilities and more to do with intense model optimization.

  While it might not be the sexy story that generates the unending wave of headlines around deep learning, hyperparameter tuning is a big barrier when it comes to new leaps in deep learning performance. In more traditional machine learning, there are plenty of open sources tools for this, but where it is needed most is in deep learning—an area that does appear to …

  Read more

• Escher Erases Batching Lines for Efficient FPGA Deep Learning

  April 25, 2017 Nicole Hemsoth

  

  Aside from the massive parallelism available in modern FPGAs, there are other two other key reasons why reconfigurable hardware is finding a fit in neural network processing in both training and inference.

  First is the energy efficiency of these devices relative to performance, and second is the flexibility of an architecture that can be recast to the framework at hand. In the past we’ve described how FPGAs can fit over GPUs as well as custom ASICs in some cases, and what the future might hold for novel architectures based on reconfigurable hardware for these workloads. But there is still …

  Read more

• Taking the Heavy Lifting Out of TensorFlow at Extreme Scale

  April 25, 2017 Nicole Hemsoth

  

  There is no real middle ground when it comes to TensorFlow use cases. Most implementations take place either in a single node or at the drastic Google-scale, with few scalability stories in between.

  This is starting to change, however, as more users find an increasing array of open source tools based on MPI and other approaches to hop to multi-GPU scalability for training, but it still not simple to scale Google’s own framework across larger machines. Code modifications get hairy beyond single node and for the MPI uninitiated, there is a steep curve to scalable deep learning.

  Although high performance …

  Read more

Previous Articles