AI
Taking the Heavy Lifting Out of TensorFlow at Extreme Scale
There is no real middle ground when it comes to TensorFlow use cases. …
MPI
AI
Scaling Deep Learning on an 18,000 GPU Supercomputer
It is one thing to scale a neural network on a single GPU or even a single system with four or eight GPUs. …
AI
Pushing MPI into the Deep Learning Training Stack
We have written much about large-scale deep learning implementations over the last couple of years, but one question that is being posed with increasing frequency is how these workloads (training in particular) will scale to many nodes. …
Code
MPI and Scalable Distributed Machine Learning
MPI (Message Passing Interface) is the de facto standard distributed communications framework for scientific and commercial parallel distributed computing. …
Code
Mitigating MPI Message Matching Issues
Since the 1990s, MPI (Message Passing Interface) has been the dominant communications protocol for high-performance scientific and commercial distributed computing. …
Compute
Intel Stretches Deep Learning on Scalable System Framework
The strong interest in deep learning neural networks lies in the ability of neural networks to solve complex pattern recognition tasks – sometimes better than humans. …
Code
A Hybrid Path to Programming Exascale Machines
As we have been describing here in detail, there is little end in sight to the train of exascale computing challenges ahead. …
Connect
Mellanox Turns InfiniBand Switches Into MPI Accelerators
One of the key themes to improving the performance of clusters running simulations has been the offloading of common routines from the central processors in the servers to accelerators in the network adapter cards that plug into the servers and that interface with switches. …
Code
Evolving the HPC Software Stack with Fresh Hardware
There is a gentle touch of hypocrisy about the supercomputing world. …
Code
Exascale HPC Needs An Application Innovation Spark
Every evolution in computing hardware brings with it big challenges for software developers. …