VMware’s Platform Revolves Around ESXi, Except Where It Can’t
August 29, 2017 Timothy Prickett Morgan
Building a platform is hard enough, and there are very few companies that can build something that scales, supports a diversity of applications, and, in the case of either cloud providers or software or whole system sellers, can be suitable for tens of thousands, much less hundreds of thousands or millions, of customers.
But if building a platform is hard, keeping it relevant is even harder, and those companies who demonstrate the ability to adapt quickly and to move to new ground while holding old ground are the ones that get to make money and wield influence in the datacenter. VMware, which started out as a company that provided desktop virtualization that allowed PCs and workstations to support multiple and often incompatible operating systems at the same time, jumped to the datacenter and built an expansive and complex platform.
Several platforms, in fact. But inevitably, all of these many different platforms – whether acquired, homegrown, or a mix of the two, always seem to come back to run atop or within the core ESXi hypervisor that made its way onto servers fifteen years ago and that has made VMware a force in the datacenter. These days, the VMware platform in its many guises usually includes NSX virtual networking and often vSAN virtual storage, and VMware has largely fulfilled its dream of creating a software-defined datacenter – what it originally called a “21st century software mainframe” back in 2009, to our amusement – from its core virtualization technology, which is used by over 500,000 customers worldwide.
By any measure, VMware has been an enormous success, and the customer and partner extravaganza being held out in Silicon Valley this week for partners and customers is mostly about celebrating that. It is hard to say for sure, but we think the VMware installed based probably looks a bit like the proprietary mainframe and midrange bases of the 1980s and early 1990s, with a representative distribution across company sizes, industries, and geographies. Our best guess is that there are maybe 50,000 mid-sized and large organizations in the world doing sophisticated IT of any fashion and scale, and that the remaining millions are using online services or a tower server tucking in a closet somewhere – or both. So only about one-tenth of the VMware base is doing distributed computing at a scale that is interesting to us at The Next Platform, and if you want to be really honest, there are probably only about 5,000 to 10,000 that are operating at anything close to scale.
That is about as good as it gets, by the way, and that massive base of small and midrange business is what makes VMware strong and formidable, from a technological and economic standpoint, just as the much larger base of small customers using IBM mainframes helped support the larger customers – spreading research, development, and overhead costs as well as seeding the market for the next generation companies because a few always break out and get a lot bigger. This is an old phenomenon, but it is also a new on. The same holds true of Nvidia’s desktop graphics card business, which is dominated by gamers and workstation users who provide the foundation for the much more expensive and sophisticated Tesla and GRID coprocessors that are generating a lot of the company’s excitement and profits.
VMware has been expecting for its core server virtualization business to start declining, and has been perfectly upfront about this. It acquired SpringSource for the Cloud Foundry platform cloud, GemStone Systems for its Smalltalk-derived development platform, RabbitMQ for its eponymous queueing software, and a bunch of other stuff that eventually got rolled into its Pivotal spinoff as a kind of insurance policy in case the ESXi/vSphere platform fell out of favor in enterprises. VMware bought Wanova and DynamicOps to shore up its cloud management capabilities, and these are now part of the vRealize management stack. And the company shelled out $1.2 billion back in 2012 to snap up Nicira, a network virtualization startup that created its own twist on the OpenFlow virtual control plane and the Open vSwitch virtual switch, now known as NSX. vSAN is something that VMware created in house, and which now has closed to 10,000 customers as of the end of the second quarter. NSX numbers were not divulged along with the most recent second quarter numbers, but as of the end of 2016, NSX had been installed at approximately 2,600 customers. Our guess is that is that the NSX customer base now numbers around 3,500.
What we can tell you is that NSX bookings grew by 40 percent in the second quarter ended on August 4 (VMware parent company Dell has weird quarter start and end dates, and they are not calendar quarters), and the top ten deals that VMware did in the second quarter all had NSX as a component. vSAN grew bookings by 150 percent for the fifth straight quarter, and when you add up all of the hardware and software and services for vSAN, this is probably a business that drives somewhere on the order of $3 billion across VMware and its partners. That is much bigger than hyperconverged storage startup Nutanix, which is probably a factor of 10X smaller. (Dell claims no favorites between VMware and its hyperconverged partner Nutanix, but you have to believe that once it bought EMC and VMware, it wants to sell VMware first and Nutanix second.)
And just in case none of this pans out over the longest of hauls, VMware has created a minimalist Linux called Photon and a container management system called Photon Controller that is based on a lightweight version of ESXi mashed up with NSX that can run the Kubernetes cloud controller that was open sourced by Google and is rapidly becoming the de facto cluster management system on the public clouds and in private datacenters. The Photon Platform won’t look and feel like ESXi to users, but deep down inside, it really is. And just in case, VMware has vSphere Integrated Containers, a variant of the OpenStack cloud controller that is implemented in ESXi virtual machines and that control freaks ESXi and its VMs.
The funny bit is that the core compute business, which is dominated by sales of the ESXi hypervisor in its many marketing guises and the vCenter controller for it, has rebounded a bit, rising 10 percent in the second quarter, and bookings for these products are now expected to be flat over the next few years and overall revenues on the rise thanks to juicy support contracts to the tune of low single digit growth. That’s pretty good for something that looks and smells like a legacy mainframe base.
In the second quarter, VMware’s revenues rose by 12.2 percent to $1.9 billion, with software license revenue up13.3 percent to $732 million. Professional services revenues (meaning consulting engagements with its biggest customers) brought in $153 million, up 14.2 percent, and the core software support services across those 500,000 customers, which gives VMware its might and its security, accounted for just over $1 billion, up 10.9 percent. Things are looking so good that VMware upped its outlook for fiscal 2018, which will end early next February. The company is now expecting to post $7.83 billion in revenue for the full year, up around 10 percent, and license revenue is expected to rise by around 10 percent as well to nearly $3.1 billion. If all goes as it has been going, VMware will break $1 billion in net income for fiscal 2018, we think, which is respectable for a software company with some pretty heavy competition, especially from direct competitors Microsoft and Red Hat and all of the public cloud providers that are indirect competitors with their own platforms even if they are also sometimes partners.
And, significantly, it will be sitting on well north of $8 billion in cash plus the $4 billion in notes it just borrowed.
Spanning The Public Clouds
Like other server infrastructure providers, VMware had a go at creating its own public cloud because the company could see the handwriting on the wall and could read it but that is a game for companies with much deeper pockets than VMware, or even Dell, which mothballed its own public cloud, can afford. (We did the math on this last fall, and showed that VMware would have to burn all of its dough to build a cloud big enough to supports its 500,000 customers – and then some.)
Aside from the usual blizzard of point release product updates that is typical for vendor events like the VMworld shindig happening this week, there was a bit of news relating to VMware and its cloud partners, which now includes Google, oddly enough, as well as Amazon Web Services, which amazingly decided to host a VMware-only public cloud on its infrastructure as part of a broad partnership agreement announced last fall.
Some sort of deal between Google seemed almost inevitable, with VMware co-founder Diane Greene, who was ousted from the company in 2008, being put on the Google board of directors in 2012 and three years later being put in charge of Google Cloud Platform. Gelsinger, the VMware CEO who used to run Intel’s server chip business and who invented the tick-tock method of rolling out Xeon processor advances, is well acquainted with Greene and was no doubt eager to spread VMware’s software across more public clouds to mitigate risk and drive revenues. The company can’t depend entirely on AWS, and in fact has thousands of partners in its vCloud Air network of cloud providers. But if you add them all up, there is no way they matter as much as AWS does and as Google Cloud Platform and Microsoft Azure will.
Google and Pivotal, the part of VMware that was spun out with the platform cloud stack based on Cloud Foundry, having been working together on “Project Kubo,” an effort to mash up the Kubernetes container controller with the BOSH software distribution management software that was at the heart of the Cloud Foundry platform cloud stack.
BOSH can deploy software to virtual machines or containers, and many flavors of either, and under the partnership announced this week at VMworld, Google is integrating Kubo, which is now known as Pivotal Container Service, or PKS for short, with its Cloud Platform public cloud and its implementation of Kubernetes, which is called Google Container Engine, or GKE for short. There is nothing too difficult about this. Kubo rides atop the GKE management APIs, much as it will ride atop the ESXi/vSphere APIs to apportion resources to Kubernetes, which in turn fires up containers. Google and Pivotal have been working on Kubo for quite a while, and the important thing is that with the tight integration, Pivotal shops who want to deploy on the latest Kubernetes will be able to get it all done by Google through Cloud Platform.
The neat bit is that Pivotal is working with VMware, its former parent and now co-member of the Dell collective, to create a private variant of the same Pivotal Container Service that will run on X86 iron and Ethernet switching that makes use of ESXi hypervisors and VMs on the servers and NSX virtual switching. Now, companies that want to have a consistent experience between a public and private cloud for container apps can partner with VMware and Google to get it. The private PKS stack will use vSAN for storage, vRealize Automation for orchestration and governance, vCloud Director for provisioning, and vRealize Operations for monitoring. (So, in theory, one could run the PKS stack on the AWS cloud slices that VMware has partnered with Amazon to create, effectively creating a clone of GKE to run on AWS bare metal iron. . . .) VMware and Pivotal are deploying engineers to keep the private PKS service in lockstep with the public on running on Google Cloud Platform, so there is no impedance mismatch on the respective stacks.
Pivotal Container Service is in private beta now, and will be available in the fourth quarter; pricing and packaging information will be available at that time.
The other important cloudy news coming out of VMware this week is that the public AWS cloud slices running the VMware stack are available and they have pricing.
The minimum cluster size on the VMware Cloud on AWS is four ESXi hosts, so you cannot just start with one, and the maximum cluster size is sixteen hosts, much smaller than the 500 servers per datacenter and 1,000 hosts maximum for a vCenter management domain. The AWS machines that VMware is selling do not have disk storage and only have flash, which is interesting indeed. (You need all-flash storage with vSAN to get some of the encryption and compression functions to work properly; disks are too slow and eat too much time and CPU.)
For the moment, there is one server configuration that VMware is offering. It is a two-socket Xeon server with 18 cores per socket with HyperThreading turned on, and that looks like a high-end “Broadwell” Xeon processor to us. This virtual machine is equipped with 512 GB of main memory and 15 TB of flash storage on the node. The pricing is as follows:
The base on demand price for this server is $8.3681 per hour, which works out to around $6,109 per month. The base EC2 machine on the AWS cloud proper that is most like the VMware Cloud variant is the i3.16xlarge instance, which has 32 cores (probably “Haswell” Xeons) and 64 threads with 488 GB of virtual memory plus 15.2 TB of flash across eight SSDs. This costs $4.992 per hour, or $3,664 per month. If you do an uplift on the compute and memory and move the hardware forward to Broadwell Xeons, we think the underlying VMware on AWS hardware in the instance is probably worth about $4,000 a month or so. That means the VMware software stack – ESXi and vSphere and NSX plus the ability to hook in all of the vRealize management tools customers already have on site to run it all – is worth about $2,110 per month. In this case, the hardware is two thirds of the system cost and the software is only one third. This is precisely the opposite of what most VMware customers experience.
Interestingly, VMware is making the case that, thanks to the ability to overprovision VMs on top of the ESXi hypervisor, the typical customer (who presumably is not doing this) can actually save money by moving to the VMware Cloud on AWS and paying for reserved instances over three years compared to their onsite VMware stacks. The math VMware is showing indicates savings of anywhere from 40 percent to 50 percent. This deserves some more detailed analysis, and if customers save that kind of money, the question is this: what impact does this have on VMware’s future revenues and profits? It looks like maybe VMware wants to make it up in volume a little bit, and it may have decided to charge a premium for on-premises software rather than the stuff it manages on behalf of customers on the VMware Cloud on AWS.
VMware will be coming to other AWS regions and instances will eventually be available in other capacities and at other prices.
A Few Other Things
There are a few other odds and ends that VMware rolled out this week. First, vSphere Integrated OpenStack has been updated to a 4.0 release based on the “Ocata” distribution of OpenStack, which came out back in March. The new VIO, which will be available on September 19, also integrates support for Kubernetes container orchestration. Interestingly, this integration is not using the Project Magnum container API that is part of OpenStack, but rather uses Docker Datacenter or Mesosphere, which certify OpenStack as resource providers, as the integration point between OpenStack and Kubernetes.
Mahesh Kumar, senior director of marketing at VMware in charge of the cloud management products, tells The Next Platform that while there are some flagship VMware customers who have deployed VIO as their production systems, for the most part customers are deploying it for their development and test environments, where programmers want to embrace OpenStack and the IT managers want to keep everything on a VMware substrate. This splits the difference.
The other thing that is new with VIO 4.0 is that it is no longer free. Starting with this release, VIO will cost $995 per server socket in a Datacenter Edition, but customers who are using VIO in conjunction with the vRealize management suite will be able to get it for $495 per socket. That is just the price of the perpetual license; reckon another 18 percent or so on top of that for annual support.
Here is the last interesting bit. VMware has been craving a place in HPC and data analytics for many years, starting way back with the Project Serengeti effort to cram Hadoop into VMs and tame that elephant a little.
VMware is well aware that academic and research institutions use open source and free software because they don’t have big budgets, and they certainly cannot afford to pay for vSphere Enterprise Plus licenses on the servers in their clusters. To that end, later in the third quarter, VMware will ship a vSphere Scale-Out Edition aimed at massively parallel HPC and analytics jobs that will have features specifically aimed at these workloads and that will sport a dramatically lower price per node. The stack will include the hypervisor plus vMotion and Storage vMotion live migration of compute and storage, the integrated distributed switch, SR-IOV virtualization for I/O, and storage APIs for linking out to file systems. Pricing has not been announced yet.