A Purified Implementation Of NVM-Express Storage

NVM-Express holds the promise of accelerating the performance and lowering the latency of flash and other non-volatile storage. Every server and storage vendor we can think of is working to bring NVM-Express into the picture to get the full benefits of flash, but even six years after the first specification for the technology was released, NVM-Express is still very much a work in progress, with capabilities like stretching it over a network still a couple of years away.

Pure Storage launched eight years ago with the idea of selling only all-flash arrays and saw NVM-Express coming many years ago, and started making plans to ensure that as the technology matured, the company was ready to embrace it, according to Matt Kixmoeller, vice president of marketing and product management at Pure Storage. The company already had its FlashArray//M all-flash array offerings in its lineup, and in March 2016 it added the FlashBlade arrays tuned for object and large file workloads. In April this year, Pure Storage added FlashArray//X, a refreshed and revamped enterprise flash array that includes not only the vendor’s new Purity DirectFlash software, but also NVM-Express DirectFlash modules.

Pure Storage is one of a growing number of vendors who are expanding their NVM-Express product portfolios, with the list including Dell EMC, NetApp, IBM and Intel. NVM-Express is following through on its promise of faster performance and lower latency, Kixmoeller told The Next Platform. But as development of the interface continues, it will continue to bring new capabilities into enterprise datacenters. NVM-Express can be used in several ways, including using it inside a storage array to improve communication and in a host to allow local-attached SSDs, he said. Over the next year or two, NVM-Express also will be stretched over a network via NVM-Express-over-Fabric, which will enable faster connectivity between servers and storage devices.

As we have written here at The Next Platform, adoption of the technology is expected to accelerate, including in the HPC space and among cloud providers and larger enterprises. During a talk at the SC17 show last month, Rick Kumar, vice president of product development at Newisys, spoke about the benefits NVM-Express will bring to the datacenter at scale, but also said that organizations will want to weigh various tradeoffs between latency, flexibility and capacity as they consider their options.

Kixmoeller says there are two broad market opportunities for NVM-Express, including to drive better performance and density in all-flash arrays.

“When we move from FlashArray//M, which is our SAS-based product, to FlashArray//X, which is 100 percent NVM-Express inside, we see latency drop by about 50 percent and we see bandwidth increase significantly, so it’s just a great tool for us to march forward on a performance path for all-flash arrays,” Kixmoeller explains. “Why that’s kind of relevant now is related the size of SSDs, where the thing that killed hard drives is starting to hurt SSDs. Hard drives died because every year they got bigger, but they never got faster, so their performance-terabyte actually went down every year. The same trend is starting to happen with SSDs, where we now have these massive SSDs – 15 TB and more – and if I go onto the market today and buy a 1 TB SSD or a 15 TB SSD, they both have the same performance. In other words, the 15 TB one is 15 times slower, so we have been able to push NVM-Express into our systems to get over that barrier, where we can shift ultra-dense flash modules but not have that kind of performance degradation.”

The other opportunity is to enable shared storage to take on a set of workloads that in the past primarily were aimed at server-centric storage.

“If you look at most of the web-scale architecture out there, you see a lot of white-box servers, commodity servers with either hard drives or direct-attached flash, SSDs or PCI-Express cards inside,” he said. “Much of the modern web runs on this architecture. What we see as an opportunity is to decouple the storage from the server architecture and bring a centralized storage device into these web-scale architectures. We call this concept top-of-rack flash. If you can put a 2U or 3U flash device on the top of every rack, and then network flash down to all the servers in the rack, with fast Ethernet and then NVM-Express-over-Fabrics, then you end up with a real nice situation with all the benefits of sharing so that you can allocate just the right amount of flash and you can service it better for the rack. But you still have something that has the performance of DAS – or pretty close to the performance of DAS, anyway – and a much better reliability profile.”

Pure Storage first introduced NVM-Express in 2015 with the launch of the FlashArray//M. The company built its own NVM-Express devices – essentially the product’s caching tier to hold the IO while deduplication and compression are underway – and developed its own protocols in its Purity software to enable a shared device that is hot-pluggable and dual-port.“Where we haven’t seen it take off broadly in the enterprise SSD space, because for an enterprise array like for storage, you need a dual-ported SSD so that the SSD can be connected to two controllers at once for reliability,” Kixmoeller said. “You also have to work through hot-pluggability and the reliability of that connection. It’s a very different approach than when you have a SSD connected to one CPU in a server where it never gets touched.”

Pure Storage is seeing business for its all-flash arrays grow, and with the FlashBlade is expanding into traditional HPC shops as well as into new, modern analytics applications. In the third quarter, the company brought on more than 300 new customers to bring its total past 4,000, a 54 percent increase from the same period last year. Pure Storage sells predominantly into cloud providers and larger enterprises, though Kixmoeller said NVM-Express eventually will make its way down to mid-size enterprises. They boast such customers as the San Francisco Giants, Mercedes, and Navitus Health Solutions. Mac Stadium, an infrastructure-as-a-service provider for Apple Mac users, is bringing FlashArray//X into the environment to address customer as-hoc demands for more capacity.

Cloud providers are particularly interested in Pure Storage’s top-of-rack flash concepts as an option to their current in-server storage.

“In these situations, it’s about making sure you achieve performance parity,” Kixmoeller said. “You’re never going to outperform the SSD that would have been right in the server, so it’s about how do you perform at right about the same level, but getting all the benefits of shared storage, which include higher reliability, lower costs because it’s centralized and shared, the ability to do data reduction, simple servicing – where you can pop out an SSD and put in a new one if something fails, rather than replacing an entire server node – and perfect allocation. Today when you put a few SSDs into as server, and maybe the application on that server uses 10 percent of that space, or maybe it runs out of space in two months and you can’t make it bigger. You see a lot of stranded storage because you don’t have the ability to vary the perfect amount of storage for the use case, where with a shared storage device you can dynamically change the amount of storage connected to each server. You can really scare storage and scale processing – server capacity – independently.”

Kixmoeller sees NVM-Express rolling out in three main waves over the next few years. A lot of single-ported flash drives will continue transitioning to NVM-Express because it’s an easy move.

“Enterprise storage arrays are going to have a tough time because it’s very hard to retrofit NVM-Express into a legacy storage array,” Kixmoeller says. “It’s really a software and a hardware problem, and it’s going to be a tough cycle for some of our competitors to bring NVM-Express out. The NVM-Express-over-Fabric standardization is going to happen over the next couple of years. That will take a bit longer because when you ship something that truly heterogeneous like that – where any server brand can connect to any storage brand through any switch – it takes time to get all the interoperability worked out in the industry. Right now, the basics work. You can pass I/O, you can reliably deliver the network, but hot-pluggability and multi-path I/O and error recovery and restarting – all the technologies to take something working 99 percent of the time to six 9s – are the ones that are really being bullet-proofed right now. It will be a path over the next couple of years to get to NVM-Express-over-Fabric.”

Pure Storage’s plan is to offer an upgrade to NVM-Express-over-Fabric in the next year to allow customers to start moving in that direction. In addition, the company is about to make its Purity ActiveCluster active-active and synchronous replication technology generally available. “The combination of NVM-Express and ActiveCluster allows us to really go after the very high end of the tier-one part of the storage market like we haven’t been able to before, so we see an opportunity to redefine tier-one storage, to replace those refrigerators full of disks with a fundamentally more efficient architecture,” says Kixmoeller.