Large language models, also known as AI foundation models and part of a broader category of AI transformer models, have been growing at an exponential pace in terms of the number of parameters they can process and the amount of compute and memory bandwidth capacity they require. …
The hyperscalers and cloud builders are the toughest customers in the IT sector, demanding the highest performance at the lowest price and an ever-improving ratio between the two. …
A massive buildout of infrastructure is happening within the datacenter walls of at least several of the hyperscalers and large clouds in the world if the financial results of Arista Networks, the upstart switch maker that has been taking on Cisco Systems in the datacenter with machines based on merchant silicon for more than a decade. …
While AI training dims the lights at hyperscalers and cloud builders and costs billions of dollars a year, in the long run, there will be a whole lot more aggregate processing done on AI inference than on AI training. …
If there is one thing that is consistently true about HPC clusters for the past thirty years and for AI training systems for the past decade, it is as workloads grow, the network becomes increasingly important – and perhaps as important as packing as much flops in a node as physically and thermally makes sense. …
A decade and a half ago, when Dennard scaling ran out of gas and many of us were starting to first think about what the end of Moore’s Law might look like should that day ever come, a bunch of us were kicking around what it might mean. …
The Hot Chips 34 conference that is normally held at Stanford University is in full swing this week, and thanks to the coronavirus pandemic is being held entirely online. …
We have been excited about the possibilities of adding tiers of memory to systems, particularly persistent memories that are less expensive than DRAM but offer similar-enough performance and functionality to be useful. …
For a company that has been so enthusiastic about designing and building its own infrastructure and datacenters, Meta Platforms, the parent company to Facebook as well as WhatsApp and Instagram and one of the champions of the metaverse virtual reality a lot of us first read about in Burning Chrome, sure has not been building its own AI supercomputers lately. …
Et tu, Meta?
Of all of the world’s hyperscalers and large cloud builders, only Meta Platforms, dominated by its Facebook and related social network businesses like Instagram and WhatsApp, is a pure-play hyperscaler. …
All Content Copyright The Next Platform