Dense, Custom Servers Are Not Always A Good Fit

Here’s a good question to ask: What happens if hyperscale customers who have been pushing their server makers to create density optimized machines move back towards designs that are more monolithic? You know, boring old rack servers, but with all the frills removed.

This could be happening, if the recent deals from Dell, which sells machines to several of the hyperscalers through its Data Center Solutions unit and which recently launched a new Datacenter Scalable Solutions unit to go after organizations that are large scale but nowhere near hyperscale, are any indication.

“When we talk about hyperscale, we are talking about the top seven companies – three are located in China and four are located on the West coast of the United States,” explains Jyeh Gan, director of product management and strategy for the new DSS unit, which was launched formally back in August but which has been doing business in stealth mode for over a year now. (In case you can’t rattle them off, that’s Google, Amazon, Microsoft, and Facebook in the US and Alibaba, Baidu, and Tencent in China.)

Dell’s DCS unit, which was formed in 2007 to make custom servers for these customers and which should not be confused with DSS, invented the four-node, 2U density-optimized machine for one of these customers (we don’t know which one), and has created other density-optimized designs to cram more CPUs and sometimes GPUs into a multinode-chassis at the behest of its bespoke systems customers.

“And some of these companies are now moving away from density-optimized machines and back towards monolithic systems,” says Gan. “They are realizing that while shared infrastructure does lower costs because you are amortizing power, cooling, and so forth across many nodes, there are additional costs that might be derived from service and support models, because it is harder to manage a shared infrastructure depending on whether they control their datacenter environment or not. I have talked to customers in China who have moved away from density optimized because the overall costs are different.”

“The DSS machines are cheaper than PowerEdge machines, and in some cases, depending on what is in the boxes, they might be cheaper than hyperscale DCS machines. Customers ask the same question: How much cheaper is this? And I tell them to really look at it.”

Not every hyperscaler is using density-optimized machinery, by the way. From pictures, Google appears to prefer rack servers with no enclosures, as it has for a decade and a half now, although the company has not disclosed much about its server designs since 2009. (A long time ago.) We see Facebook’s designs through the Open Compute Project, which include a mix of rack and density-optimized gear, and we have very little idea about what Amazon is doing. (We do get hints from time to time, like at the re:Invent conference two weeks ago.)

One of the problems with density optimized machines is that the relatively small nodes that go into the enclosures limits the CPU, memory, and peripheral options customers have. The power density of the racks can also be quite high, which can make providing electricity and cooling to them a challenge as well. As Gan, who started out as a design engineer at Dell in 1999 for PowerEdge rack and blade servers, succinctly put it: “If you have a density-optimized machine, but you only fill it halfway up in the rack because of power issues, what is the point?”

That is an interesting tidbit of information and perhaps goes a long way towards explaining the fact that the first three servers to come from the new DSS unit are rack machines, not density-optimized ones with multiple nodes in an enclosure. (You almost have to call the new DSS storage server, which crams 92 drives in a 4U space, a density-optimized server, but this one is dense when it comes to storage more than for compute.) The new DSS machines are being rolled out at the Dell World conference this week in the company’s hometown of Austin, Texas.

Taking On The Next 1,000 Server Scalers

DSS has been running behind the scenes for a little more than a year, and is targeting the next 1,000 large-scale customers in the world, who buy machines by the thousands and who might have many thousands to tens of thousands of servers scattered around the world in their datacenters.

The target that Dell is aiming at with DSS is quite large – on the same order of magnitude that the infrastructure supplier has tackled with its DCS group, which was founded in 2007 and ramped to over $1 billion in revenues within a few years and had shipped over 1 million custom machines through 2012. (Our guess is that DCS is close to shipping another 1 million machines since then, but it is hard to say for sure.) This hyperscale part of the server market drives somewhere between $7 billion and $8 billion a year in sales (across the industry, not just for Dell) and is growing at a faster rate than the overall server market, which is creeping along with revenue and shipment growth in the single digits most quarters. But the large-scale portion of the market, which Dell reckons has a total addressable market of around $6.6 billion, right now, according to Ashley Gorakhpurwalla, vice president and general manager of Server Solutions at Dell, is growing at a 14 percent compound annual growth rate – about three times the rate of the overall server market. The core enterprise portion of the systems market is still much larger, at somewhere between $25 billion and $26 billion a year in revenues, but has modest growth (maybe 5 percent) at best and flat revenues at worst, depending on the quarter and the year.

dell-dss-server-landscape

The DSS unit aims at tackling that faster-growing large-scale systems market, where companies want to emulate some of the minimalist practices of hyperscalers without having to design their own machines and manage their own supply chains, as all of the hyperscalers do. (Even Microsoft, which buys machines from Dell, Hewlett-Packard, and Quanta, secures its own components for the machines that these companies in turn use to make the systems Microsoft needs to build out its cloud.) The DSS machines are not painted, they don’t have a lot of the serviceability features of the regular, enterprise-grade PowerEdge machines, and they often come with a more basic warranty.

DSS had already amassed 200 customers when it was still operating in stealth mode through its launch in August, and had exhibited 460 percent year-on-year revenue growth in its most recent quarter. Gan tells The Next Platform that since DSS came out of stealth in August, it has engaged with more than 50 more customers and its revenue growth rate is now up 800 percent, year on year. “We are growing very quickly, and we are definitely the fastest growing business in Enterprise Systems Group,” says Gan.

How many machines DSS has sold and what amount of revenue remains a mystery, however. Dell is privately held, and even when it was a public company it only hinted at the size of the DCS unit. But we have some hints. Gorakhpurwalla, who runs all of Dell’s server operations, told us back in August that the desire was to have DSS be as big as DCS within a year or so (in terms of revenues), and it looks like the company is on track to break through 1,000 customers with DSS within the next two years, when Dell expects DSS to drive more revenues and shipments than DCS does today. That will give Dell a big share of the kiloscale and megascale parts of the datacenter server space.

The DSS machines have all but the bare necessities removed from them and assume, as do the DCS machines used by hyperscalers, that the resiliency and management of the systems will come from the software stack, not from specialized controllers or redundant components.

Dell started DSS doing deals with telecommunications companies, hosting companies, service providers, and oil and gas companies, and initially only in North America. But in recent months, Dell has expanded DSS engagements out to research and development centers and financial services firms, who are also often running clusters and want minimalist machines so they can save a little dough. Dell has recently expanded the sales footprint to Europe and Asia/Pacific-Japan. Gan says that most of the revenue for DSS is still coming from North America, but that Asia has quickly become about a third of the business with North America making up the rest. But then again, Gan says with a laugh, most of the companies that are headquartered in North America are global and they have datacenters around the globe, so where the headquarters is located is not necessarily a good proxy for where the DSS machines have actually been installed.

Dell thinks that big companies that have been trying to run their own supply chains, or have original design manufacturers do it on their behalf, will want to take a second look at DSS iron if they have the scale to warrant an engagement.

“The DSS machines are cheaper than PowerEdge machines, and in some cases, depending on what is in the boxes, they might be cheaper than hyperscale DCS machines,” says Gan, adding that Dell is not comfortable sharing pricing information publicly on DSS or DCS iron. But everyone wants to know pricing on DCS and DSS machines compared to a regular PowerEdge or a PowerEdge-C, the latter being an earlier generation of hyperscale-inspired machines aimed at even higher volumes. “Customers ask the same question: How much cheaper is this? And I tell them to really look at it. Say they buy from some whitebox vendor, and then they tell me they have to buy spare machines because they don’t get the service and support they need, and they buy components from parts distributors, and then third party warranty and support for all of this stuff. If you add all of that up, maybe it is cheaper and maybe it is not, but the DSS machines we are selling are suddenly much cheaper and it is so much simpler to just buy it.”

You can tell now why Dell has been tepid about most Open Compute designs except the one from Microsoft, called the Open Cloud Server, where it gets a big chunk of DCS change when it wins its portion of the deals.

The Feeds And The Speeds Of The DSS Machines

The DSS machines announced this week are all based on Intel’s “Haswell” Xeon E5 v3 processors, and they have very precise and somewhat limited operating system support. The machines have a very basic baseboard management controller and do not have Dell’s iDRAC server controller, used in its flagship PowerEdge machines, as an option. Dell also has tailored support for customers, including scale optimized break/fix services that take into account that the large organizations that are buying DSS machines usually have the capability to swap out broken machines with ones they have in stock and then do their own repairs.

dell-dss-servers

Gan calls the DSS 1500, 1510, and 2500 machines “blank canvasses” that companies can use to deploy their applications. The DSS 1500 is a two socket machine with 16 memory slots in a 1U pizza box, while the DSS 1510 is a single-socket machine with eight memory slots; both support either four 3.5-inch or eight 2.5-inch drives, and they can be enterprise SAS, nearline SAS, or SATA disk drives or SATA SSDs. They have a two-port 1 Gb/sec Ethernet controller and an optional 1 Gb/sec Ethernet port for management. The DSS 2500 takes a two-socket Xeon E5 motherboard and puts it into a 2U chassis that has room for a dozen hot-plug 3.5-inch drives in the front plus two internal 2.5-inch drives for operating system storage. The machine has sixteen memory slots for a maximum of 512 GB of capacity, and has two 1 Gb/sec Ethernet ports with the optional management port and four PCI-Express 3.0 slots (one x16 and three x8).

The DSS 1500 on the left, and the DSS 2500 on the right
The DSS 1500 on the left, and the DSS 2500 on the right

Microsoft Windows Server 2012 R2 and Red Hat Enterprise Linux 7.0 are the only two operating systems supported on these machines. Operating systems are not pre-installed on the machines – it’s DIY.

The most interesting machine that Dell is launching in its new large scale lineup is arguably the DSS 7000, which is a storage server. This 4U enclosure has room for 90 3.5-inch disk drives, which can be a mix of SAS, nearline SAS, or enterprise SATA disk drives or SATA SSDs. The whole thing weighs in at around 285 pounds, so don’t try to lift it on your own. The fattest drives available in this machine are 8 TB SATA drives, and that makes 720 TB per unit and up to 7.2 PB in a single rack.

The DSS 700, with its server nodes pulled out of the back
The DSS 7000, with its server nodes pulled out of the back. Disk drives slot in from the top.

This being a storage server and not just a dumb JBOD box, the DSS 7000 has two half-width server nodes that tuck in under the disk drives and would be used to run workloads on the box. The server nodes come with Avago MegaRAID 9361-8i or PMC Sierra PMC8805 SAS and SATA disk controllers, and have a dozen memory slots each that top out at 384 GB each. The two internal 2.5-inch drives are used to boot operating systems, and these are limited to Red Hat Enterprise Linux 7.X, Microsoft Windows Server 2012 R2, Ubuntu 14.04 LTS, and SUSE Linux Enterprise Server 11 SP3.

The DSS 1500 and DSS 2500 servers have already started shipping and the DSS 1510 server and DSS 7000 storage server will ship next month.

Sign up to our Newsletter

Featuring highlights, analysis, and stories from the week directly from us to your inbox with nothing in between.
Subscribe now

5 Comments

  1. Dell DSS 7000 may be interesting if – and only if – it’s reasonably priced in barebone form and it’ll work with regular 3.5″ SAS3 hard drives from Seagate or HGST (because Dell-branded 3.5″ hard drives are ridiculously overpriced). And if not, Supermicro and Quanta both offer very similar storage servers (slightly less dense – but rack space doesn’t cost all that much).

    • In order to support HA storage implementation it would be nice that DSS 7000 can share JBOD drives between two server nodes, or at least to have an option for simple JBOD zone import.

  2. Shared components (like FD332 HDD tray in FX2 architecture) may pose some disadvantage in service deployment. For example, if service has to be deployed as highly available (HA) – it must be deployed on servers in different chassis, because shared component failure can affect multiple servers within the same chassis.

  3. Dell has it figured out, their high density solutions are 2nd to none and their impact on lower overall TCO with these solutions are excellent. (90) 3.5″ in a 4U space, amazing, without the quality concerns that come from some of the vendors mentioned in the article. I hope they expose more of the DCS solutions through the DSS business because they have some awesome product that’s limited to only a select few.

Leave a Reply

Your email address will not be published.


*


This site uses Akismet to reduce spam. Learn how your comment data is processed.