How In The Hell Will Any Cloud Ever Catch AWS?
April 9, 2015 Timothy Prickett Morgan
What doubles every 18 months or so and drives the IT industry? Well, it used to be Moore’s Law, which has slowed. We may have a new proxy for progress: the customer count on the Amazon Web Services public cloud, which has topped 1 million and is still growing sharply.
AWS has such a lead in terms of scope and scale on its many competitors that it is hard to imagine that any of them will ever catch up. Google and Microsoft have such deep pockets and such a deep bench of technical talent that they have a chance to get a chunk of the public cloud spending, but all others – including IBM SoftLayer and Rackspace Hosting – will have a much tougher time doing anything but carving out their niches.
This fact was in evidence, yet again, as AWS hosted a summit for customers and partners in San Francisco, and Andy Jassy, senior vice president in charge of the cloud subsidiary of the online retailing giant, trotted out some statistics showing just how fast AWS is still growing after its ninth year in the market. This growth shows what an advantage that first mover status can be in any market.
Ironically, it was a little bit hard to catch everything Jassy said because Amazon had some big issues with live streaming the summit keynote to those of us watching from the web. The summit keynote started fifteen minutes late because of technical issues, and then for most of the first hour it was ridiculously bad. It is a pity that AWS did not have access to lots of computing and network capacity with a global content delivery network that could feed the thousands of people trying to hear what Jassy had to say. Oh, wait. . . Anyway, we managed to pull a few screenshots despite all this, showing the growth for AWS on a few metrics for key services over the past several years. Sadly, not the customer count chart, which we saw flash briefly and then jitter away.
Here is what the S3 growth looks like through the end of 2014:
This is not a measure of the capacity in the worldwide object count in the S3 object storage platform – a figure that AWS used to give out in days gone by – but rather a measure of the bandwidth into and out of all of the S3 service on a daily rate. AWS added automatic deletion or movement of S3 files to its Glacier cold storage two years ago, and that has no doubt had an impact on the object count inside of S3, and intentionally so. Aggregate S3 bandwidth might be a proxy for the actual amount of data in the object storage, but AWS is not saying. The point is, the amount of S3 bandwidth was more than double in the fourth quarter of 2014 than it was in the prior quarter.
EC2 instance growth, as measured in normalized instance hours per week, almost kept pace with the S3 growth:
That EC2 usage doesn’t tell us how many instances that Amazon has running at any given time, and it surely doesn’t tell us how many servers that Amazon has. But a presentation last fall at the AWS re:Invent conference by James Hamilton, the distinguished engineer at the cloud subsidiary that steers its system architecture, gave us some clues into how many servers Amazon might have.
Amazon has eleven regions – soon to be twelve, Jassy said today during his keynote – and a total of 28 availability zones when he spoke and now 29. Hamilton said last fall that every region has no fewer than two availability zones and each availability zone has at least one datacenter (usually more, and some as high as six datacenters per zone). Then Hamilton dropped this little tidbit of information that most of the Amazon datacenters have somewhere between 50,000 and 80,000 servers. If you do some math on that, you get a server count that ranges from a low of 1.4 million machines (that’s 50,000 machines each with one datacenter in each availability zone) to a high of 5.6 million machines (if you assume 65,000 servers per datacenter and an average of three datacenters per zone). It is hard to say what the number is, but it is a big number. (Gartner calculated it was around 2.4 million last year before Hamilton have his speech, and it is my gut instinct that the number is above 3 million. Gartner also said last year that AWS has five times the cloud capacity (presumably meaning servers and storage) as the aggregate from the fourteen other cloud suppliers.
What Amazon has said for some time is that it fired up enough machinery every day to run an Amazon online retail business from 2004, back when it was a $7 billion company. Every day.
This is the crux of the advantage that AWS has.
Yes, custom servers, storage, and networks provide a competitive advantage, but it is more than shaking out efficiencies from machinery and squeezing suppliers in the supply chain. (Something that Amazon, the retailer has proved masterful at.) AWS started eleven years ago, and launched nine years ago with the EC2 service and a single instance. And now, it has a breadth of compute, storage, and network services for raw infrastructure and dozens of platform services for storing and manipulating data on top of that for those who don’t want to build such services themselves. Jassy said the combination of depth and experience at scale are also key factors in the exponential growth of AWS, which now has over 1 million active customers, doubling every 18 months or so.
Like Moore’s Law used to and doesn’t much anymore.
As Jassy put it, AWS doesn’t have one type of compute, it has many, with varying of memory, storage, and I/O capacity and even GPU-augmented compute. (Possibly FPGA augmentation soon, too, if those devices take off as we expect them to as accelerators.) It doesn’t have one type of object storage, but three. It doesn’t just have one database service, but many. And so on. “It is not just how many services we have, but how many features we have compared to other technology providers,” Jassy said, adding that you can’t just compare cloud services as if they were all sliced from the same dough with the same cookie cutter.
And then it circles back to the number of customers. “AWS has a lot more customers than any of the other cloud providers, which means we are working with large enterprises and agencies,” he explained. These customers have tough problems and they want to solve them and this drives innovation that other customers benefit from. “There’s a real network effect when you are AWS.”
Well, there is a network effect for all technology providers, so this is not actually unique to AWS. What can be honestly said is that the massive head start that AWS has after a being in the field for more than a decade and its relentless pursuit to innovate and radically upset the IT business as we know it is why customers are flocking to AWS with their workloads. That exponential growth in customers has its limits, of course, because there are only so many people, companies, and workloads in the world. But it is hard to argue that the AWS business, which is probably above $4 billion in revenues, doesn’t have a lot of years of growth ahead of it. (Amazon said recently that it will start reporting financial results for AWS.)
“We are much better at this than we were nine years ago,” Jassy said. “There is no compression algorithm for experience.”
(Hopefully there is one for the webcasting service AWS uses for its next keynote.)
Add it all up, and this is why Jassy and the other top brass at Amazon think that in the fullness of time, AWS could be a larger and more disruptive business than the Amazon retailer, which is ten times bigger now than it was in 2004.