The idea of a “private” file system—one that runs within a user’s specific job and is tailored to the I/O requirements of a particular workload—is not necessarily new, but it is gaining steam given changing hardware capabilities and workload demands in large HPC environments. …
When it comes to parallel file systems, few people understand the evolution of challenges better than Sven Oehme, who was part of the original team at IBM building GPFS. …
As we well know by now, workloads at supercomputing sites large and small are changing with the introduction of machine learning and more complex applications that require both large and small files. …
NVM-Express has been creating a quiet revolution in servers for several years now, providing a way for flash storage to bypass the traditional storage stack and the limitations of interfaces such as SATA and SAS, and instead pipe data directly into and out of the CPU through the high-speed PCI Express bus. …
It is hard to say for sure, but there is probably as much aggregate computing capacity in the academic supercomputing centers of the world as there are in the big national labs. …
When IBM’s Summit supercomputer was officially unveiled in June, some hailed it as the first exascale system because of its peak performance in applications that made heavy use of GPU acceleration at low precision. …
Now that deep learning at traditional supercomputing centers is becoming a more pervasive combination, the infrastructure challenges of making both AI and simulations run efficiently on the same hardware and software stacks are emerging. …
It is safe to say that a little more than a decade ago, when the clone of Google’s MapReduce and Google File System distributed storage and computing platform was cloned at Yahoo and offered up to the world as a way to transform the nature of data analytics at scale, that we all had much higher hopes for the emergence of platforms centered around Hadoop that would change enterprise, not just webscale, computing. …
Many programs have a tough time spanning across high levels of concurrency, but if they are cleverly coded, databases can make great use of massively parallel compute based in hardware to radically speed up the time it takes to run complex queries against large datasets. …
Perhaps the most important decision that any company will ever make is how they intend to structure and store the information they will preserve to encapsulate the goods and services they provide to their customers. …
All Content Copyright 2017 The Next Platform
