Hybrid Disk/Flash Storage Isn’t New, But It Is Getting Better

Five years ago, Western Digital, known for its hard disk drive (HDD) storage technologies, doled out $19 billion in cash and stock for SanDisk and its solid state drive (SDD) product portfolio, giving it deep expertise in the non-volatile flash memory space at a time when enterprises were looking for expanded storage options in the wake of the rise of the cloud and the edge.

At the time the deal was announced in 2015 — the acquisition closed a year later — then-CEO of Western Digital Steve Milligan boasted that the “combined company will be ideally positioned to capture the growth opportunities created by the rapidly evolving storage industry.” The company also noted that with SanDisk in the fold, the combination of the two will enable Western Digital to “vertically integrate into NAND, securing long-term access to solid state technology at lower cost.”

The deal gave Western Digital a much broader portfolio of technologies and expanded reach beyond HDDs. Earlier this month, the company announced that in the most recent quarter, revenue jumped 15 percent year-over-year, to $4.9 billion. In its Data Center Devices and Solutions business unit, Western Digital reached a record shipment of more than 104 exabytes in capacity enterprise hard drives, with almost half of those shipments centered on the company’s 18-terabyte hard drive. Demand for enterprise SSDs also grew, with the company seeing growing interest in a broad range of companies as well as cloud providers.

All this comes as storage demand means strong growth in both HDD and SSD markets. IDC in June said HDD petabyte shipments will grow an average of 18.5 percent per year between 2020 and 2025, with average capacity per drive growing an average of 25.5 percent. SSD unit shipments will jump 7.8 percent a year through 2025 and revenue will increase 9.2 percent, reaching $51.5 billion in 2025. SSD capacity shipments will jump 33 percent a year.

“While the client HDD market continues a long-term secular decline due to rising SSD attach rates, the COVID-19 pandemic has over the near term increased the demand for certain types of HDDs, particularly mobile HDDs as well as capacity-optimized HDDs,” said Edward Burns, research director of hard disk drive and storage technologies at IDC. “The demand for storage capacity continues to grow at a steady pace as the world creates and stores more and more data.”

The rapid increase in the amount of data being generated has been well documented and something Western Digital is well aware of.

“It’s being created from various sources,” Ravi Pendekanti, senior vice president of product management and marketing of HDD at Western Digital, tells The Next Platform. “It’s come up from the likes of IoT with 5G. Machine learning is also playing a huge role, if you look at some of the things that we hear about, from blockchain to the advent of 5G sensors and IoT stuff. Beyond that, there are a lot of regulatory needs that are coming up. Also, we are storing a lot more data; we will probably store 2.5 times more data this year compared to the last, probably the same next year.”

At the same time, Western Digital over the past five years has greatly built up the IP it had created on both the HDD and flash sides of the spectrum. On the HDD side, the company rolled out its HelioSeal helium HDDs and double- and triple-stage actuators, and over the past year has been pushing its ePMR technology, which puts a bias current into perpendicular magnetic recording technology to reduce jitter, or distortion. With flash, the company was offering such technologies as multi-bit cell architectures and 162-layer 3D flash memory.

At the recent HDD Reimagine event, the company unveiled a new technology called OptiNAND, which integrates an iNAND Universal Flash Storage with an HDD and also comes with revamped firmware algorithms and system-on-a-chip (SoC). The technology — which leverages both Western Digital’s long experience with HDDs and the flash capabilities inherited in the SanDisk deal — is being sampled now with a range of select Western Digital customers and products based on OptiNAND and further enhancements to the technology are on the way.

“It is the blending of these two innovations that really shows that we said we could come up with something pretty unique in the industry,” says Pendekanti, whose background includes executive positions with Dell Technologies and Oracle. “OptiNAND technology really brings both those two worlds together and builds on all the synergies between the two sides. Engineering teams have spent quite a bit of a time trying to see what this means, and it all points to what our customers are asking for, which is higher capacity, better performance and higher reliability.”

Such demands are driving the need for new and enhanced storage technologies. Western Digital has shingled magnetic recording (SMR) devices — which are designed to improve capacity while keeping down the cost-per-terabyte — that can support 20TB, but organizations are going to push for devices that can hold up to 50TB and more, he says. With OptiNAND, the enhanced algorithms necessary to enable the blending of flash and spinning disks leverage a greater amount of metadata being offloaded to the iNAND, which drives more tracks per inch (TPI) and expanded areal density. The company expects to reach 50TB by the second half of the decade.

The samples shipped to some organizations are nine-disk, 20TB ePMR flash-enhanced drives with the OptiNAND technology.

The enhanced firmware also requires fewer adjacent track interference (ATI) refreshes and reduces the need for write cash flushes in write cache-enabled mode and almost 50 times more data can be retained if power is turned off.

The new architecture will provide 2.2TB per platter, pushing ahead with gains made on the company’s ePMR technology.

Pendekanti says Western Digital engineers were able to learn the lessons from the development of hybrid storage drives by most storage vendors over the past decade.

“The thing that really occurred was when you go back and look at the hybrid drives, based on the user data classification, if it was warm data, for example, or hot data was typically written on to flash,” he says. “User data was being written onto flash, where if it was warm or cold data — based on the classification — you could write it down to your disk platters. The key thing here is we do not hide any user data onto flash. The intention is not to try to figure out what kind of data it is and then drive to two different recording mechanisms. There’s only one recording mechanism — that is still your hard drive mechanism. What we are doing is making sure that we bring all the attributes we talked about with flash to enhance it and ensure that we are able to get better TPI, improve the ATI, making sure that the metadata is written onto the flash, and making sure that because of that, it adds to the performance implications and improves the performance and also improves the reliability. We learned a lot from some of the pitfalls of the past and [those lessons] went into the design.”

The select customers sampling the OptiNAND drives are mostly hyperscalers and enterprises, reflecting the increasingly similar workloads that both are running as data amounts grow and new workloads like artificial intelligence and machine learning emerge. The huge amounts of data being consumed and stored — remember the 104 exabytes of data being deployed on ePMR — is coming from both hyperscalers and enterprises, he says.

The evolving regulatory environment — where more rules around the world are being placed on what data can be stored where — also is factor in this.

“The regulatory stuff is becoming important to them, so that’s also driving up the capacity,” Pendekanti says. “One organization was talking about how Germany as a country doesn’t allow any data that’s been created to be stored outside the boundaries of that particular nation. When you look at this as more of the regulatory stuff is coming in play, you’re also seeing more of islands of data being created, which is going to give an interesting set of dynamics in the future.”

Western Digital is going all-in with OptiNAND. The technology is going to span multiple generations of products, which is why the company is introducing it before it finds it way into devices. Enhancements on the flash side will ensure that the storage technology can continue to be integrated into HDDs, he says.