We have been making the case in this three part series that vertical integration is becoming more popular in modern datacenters. (See the links at the bottom of this story for those two prior and interrelated stories.) And in this final installment, the subject turns to AWS Outposts, which we feel are the on-premises shard of public cloud that Microsoft’s Azure Stack should have been but wasn’t.
And that is because Azure has not integrated its Catapult SmartNICs with Azure Stack as Amazon has done with its Nitro devices inside of Outposts. The need for a SmartNIC in on-premises clouds is why SmartNIC upstart Pensando as well as its rivals who are watching what AWS has done with Nitro is so interesting.
At its re:Invent 2018 event, AWS announced its intent to launch Outposts by the end of 2019. At re:Invent 2019, AWS announced that it is now actively selling its AWS-native Outposts.
AWS Outposts are built using AWS third-generation Nitro SmartNIC as the foundation for rack-level networking. Outposts are an extension of a customer’s Amazon Virtual Private Cloud (VPC) in the closest AWS region to each customer. Outposts customers should be able to seamlessly connect from their AWS Outposts to the rest of their applications or to any other AWS service in an AWS public cloud region.
The first generally available AWS Outposts products will support these instance types: M5.12xlarge, M5.24xlarge, C5.24xlarge, R5.24xlarge and G4dn.metal, with I3en is listed as a future type.
Most Outpost instance types (M5.24xlarge, C5.24xlarge, R5.24xlarge and I3en) support EFA in the cloud. Outposts will also support AWS’s upcoming G4dn type family (not in production yet), based on a server containing eight of Nvidia’s latest Tesla T4 GPU accelerators.
AWS offers Outposts in two variants:
- AWS-native allows customers to use the same APIs and control plane on Outposts as they use in AWS public regions
- VMware Cloud on AWS Outposts is open for beta program signup now (“nominations” with no guarantee of acceptance) but will not deploy with production availability and SLAs until sometime in 2020. We’ll shelve discussion of legacy lift-and-shift of VMware hosted applications on AWS Outposts for now.
AWS says that AWS-native Outposts can be now installed and operated in the following countries: United States of America, all EU countries, Switzerland, Norway, Australia, Japan and South Korea.
Also, AWS has worked with both Cisco and Juniper to directly connect AWS Outpost’s built-in top of rack switches to enterprise core data center switches from both vendors. This makes data center integration straightforward for on prem Outpost deployments in most enterprise IT shops.
How Do AWS Outposts Stack Up To Azure Stack?
At a first glance, AWS-native Outposts appear to be similar to Microsoft’s Azure Stack offerings. But that is not the case. We took a deep dive into Microsoft’s Azure Stack a couple of years ago and concluded that “Azure Stack will eat enterprise private cloud…”
To quote further: “The near ubiquity of Microsoft’s enterprise presence – Microsoft owns commanding high-ground in operating systems, development tools, applications, channel partnerships and developer mindshare – gives Azure Stack a private cloud market potential that OpenStack and other private cloud competitors can only dream of.”
We missed the mark. What happened to Azure Stack and what’s different about AWS Outposts today?
|Component||Microsoft Azure Stack||AWS Outposts|
|Stack cloud software design||Subset of Azure core services||AWS core services|
|Stack hardware configuration||Azure specified||AWS defined|
|Stack hardware implementation||OEM gear||AWS cloud gear|
|Management layer||OEM||AWS Nitro|
|SmartNIC||No *||AWS Nitro|
|Rack switches||OEM gear or partners||AWS gear|
|Sold by||OEMs and Microsoft||AWS|
|Hardware fulfilled by||OEM||AWS|
|Hardware managed by||Customer||AWS|
|Stack variant for OEM HCI hardware||Yes, Azure Stack HCI||No|
HCI stands for hyper-converged infrastructure, and it means putting a virtual storage area network on the same cluster that is running virtual compute. It is essentially OEM blade or modular architectures with a little cloud architecture influence sprinkled in. Azure Stack HCI (previously called Microsoft Windows Server Software Defined) is compatible with Azure and Azure Stack only at a Hyper-V client level.
In a nutshell, Microsoft had the right thought about using the same code base for Azure cloud and for Azure Stack. However, there are three root causes for Azure Stack lack of traction:
- The early decision to enable OEM hardware without mandating use of its Catapult SmartNIC introduced a host of verification and scaling challenges.
- A later decision to enable Azure Stack HCI to use OEM HCI gear without actually using Azure Stack. Instead, Azure Stack HCI enables customers to use Azure-compatible containers.
- The early decision to intentionally limit market potential to a small market not easily addressed by OEMs or by public clouds at the time – intermittently connected or air-gapped (disconnected from public internet) private cloud deployments.
Azure Stack success has been hobbled by these decisions. AWS took notes and then bypassed all these challenges.
- Nitro enables all of AWS’s modern service stack, so AWS included it from the start.
- AWS sells and fulfills every bit of the solution itself, including delivery, from the start.
- Without partners, AWS Outposts is free to target all of its potential market, like any other well-managed and greedy (albeit large) startup.
Here is the basic shape of the Outpost announcement:
And here is the list of supported AWS services on Outposts and the countries where they are supported:
We believe that Microsoft Azure Stack has a chance to compete for at least part of the private cloud market. But to do so, Azure Stack must redefine itself to mandate use of the Catapult SmartNIC, whether all its OEM partners are willing to accept that change or not. To do so, Microsoft would have to expose Catapult’s features to OEM developers. The answer to maintaining some level of secrecy about its internally deployed Catapult infrastructure capabilities would be to isolate a subset of Catapult features for public exposure in OEM’s deployments of Azure Stack and manage that singular set of differences over time.
AWS now has the ability to deliver, manage and invoice services on a managed slice of its public cloud infrastructure hosted in a customer’s on prem data center. This is not only a direct threat to Microsoft Azure Stack, but it is also a direct threat to the server OEM product lines at Cisco, Dell, HPE, Huawei, Lenovo and the rest. How many existing business sectors have lost the bet that they can compete effectively with Amazon for delivering efficiently fulfilled products?
Porting existing HPC applications to AWS public cloud using MPI is possible today. Using AWS Outposts as a target requires only that AWS support EFA in Outposts in the same manner it is supporting EFA in its public cloud. Given that EFA is supported on all of the processor-only instance types available in Outposts today, that doesn’t seem like a stretch, for the right customer or the right price.
Likewise, Outpost support for P3dn.24xlarge or EFA support for G4dn.metal would simply be a matter of qualification and perhaps up-sizing rack power delivery for HPC and deep learning training applications. AWS Outpost enablement for EFA and bigger, badder GPUs, and therefore private HPC clouds, is only a matter of time.
Plus, customers get Nitro-enabled network encryption and decryption, and sole-tenant server isolation, in their on prem private cloud.
It’s starting to look like customers can keep secrets in AWS Outposts while not having to refactor applications or sacrifice bare metal speeds.
Remember, Alibaba Cloud technology development and deployed capabilities are tracking AWS. There is nothing discussed above that Alibaba Cloud could not also implement with its X-Dragon SmartNIC. While the technical hurdles may be high for the general competitive field, Alibaba Cloud will have no trouble following.
If rack-level hardware/software codesign was the limit of AWS’s thought process for Outposts, it’s still a compelling solution. But it’s not the limit, not by a long way.
Cloud Networks Also Connect datacenters
AWS also just launched its Local Zones service. AWS claims that Local Zones will provide single-digit millisecond latencies (sub-10 millisecond) from an area much smaller than a typical region directly to core services in a specific region.
AWS Local Zones are intended to provide very low latency for high performance applications in media and entertainment, ad-tech and machine learning – aiming at the center of LA’s content production and advertising industries. These applications are adjacent to many HPC applications. Local Zones is also aiming at latency-sensitive EDA applications in the southern California tech industry, which is arguably a legitimate HPC application.
The first AWS Local Zone is “generally available by invitation” in Los Angeles and nearby southern California locations. That is a weird way of saying that it is in a curated, limited-availability beta test, because if it were really generally available, then anyone with an AWS account in LA could sign up for an AWS Local Zone with a production SLA and posted pricing. The Los Angeles Local Zone (us-west-2-lax-1a) is connected to AWS’s us-west-2 (Oregon) region over AWS’s high speed private backbone network. Sometime in 2020, AWS promises to open a second Local Zone in Los Angeles (us-west-2-lax-1b).
AWS says that in early 2020 Local Zones will really be generally available in LA. Customers will then be able opt in using their AWS account console, command line interface or by API call.
However, Los Angeles is not the only current AWS Local Zones deployment. AWS created a flavor of Local Zones specifically tuned for telco needs, branded AWS Wavelength. AWS Wavelength will enable regional 5G networks to access AWS core regional cloud services at the same sub-10 msec latencies as the generally accessible AWS Local Zones service.
At re:Invent 2019, AWS positioned Local Zones as a service for customers in a particular geography but not on premises. However, there are no technical limits to doing so. These first AWS Outposts and Local Zones product offerings are just the beginning of AWS’s learning curve to bridge cloud to enterprise on prem as part of a single massively distributed system.
So, it is really not a stretch to imagine a dedicated HPC variant of Local Zones that pulls together AWS Outposts designed for HPC with AWS public cloud HPC services via a low-latency, high-bandwidth access point.
Alibaba Cloud has already created an adjacent market with its China Gateway program, which enables international companies to create a scalable online presence in China. While its Express Connect and VPN products advertise low latencies, we guess that AWS Local Zones will push Alibaba Cloud to decrease latencies even more.
What About Data Gravity?
The last part of the equation remaining to bridge HPC or other applications with high data gravity is data egress pricing. In this case, cloud data egress pricing is a somewhat artificial construct intended to keep customer data in a cloud, where processing will be performed and invoiced by the cloud. But AWS Outposts customers will pay AWS for processing regardless of whether their data resides on prem or in the cloud.
Who wants to place a bet that AWS Outposts customers won’t eventually receive preferential data egress pricing or maybe AWS cloud internal data transfer pricing for AWS cloud data egress to an on prem AWS Outposts deployment? Any takers? Any takers at all…?
The SmartNIC is the critical technology for deploying shards of public cloud into enterprise IT data centers
Why would AWS stop after reinventing the HPC market? Why wouldn’t AWS also try to reinvent private cloud computing? As in “all of it…”
AWS Outposts is everything that Azure Stack should have been. All things being equal, Microsoft would have a tough road ahead if it wants to stay in this business. However, if anything can galvanize its Azure Stack OEM partners to link arms, close ranks and actually sell Azure Stack deployments (and not Azure Stack HCI), it will be the threat of AWS selling directly to enterprise customers.
Alibaba Cloud has the technology and resources to catch up quickly after examining how AWS manages the first few quarters of sales.
Google Cloud is not prepared for this level of enterprise IT engagement. Creating a better qualified enterprise sales force will not help here, this is a technology and fulfillment battlefield.
Intel might get its act together, but it will not be in 2020 given they are just staffing up a design center to address this market.
Pensando has already published two important bits of sales collateral:
- Distributed Services Platform for Cloud Providers
- Distributed Services Platform for the Enterprise
We’ll be very surprised if there is not a plan to bridge the two, given Pensando’s founders’ collective experience. We believe Pensando is positioning to supply smaller, regional cloud providers with similar capabilities. With Mellanox Technologies off the playing field until the Nvidia acquisition closes and Intel going back to basics, Pensando may have a significant head-start on the merchant SmartNIC market.
Hewlett Packard Enterprise led Pensando’s series C funding round ($145 million, after two previous rounds totaling $133 million). Watch this space, it is sure to be interesting. It’s a smart bet on HPE’s part.
AWS Outposts may seem like a niche play. It is not. The combination of Outposts with AWS Local Zones and AWS partnerships with Cisco and Juniper signals the start of AWS’s end-game for cloudifying enterprise IT. And the fundamental enabling technology enabling this strategy is a SmartNIC.
Paul Teich is an incorrigible technologist and principal analyst at Liftr Insights, covering the emergence of cloud native technologies, products, services and business models. He is also a contributor to Forbes/Cloud. Paul was previously a principal analyst at Tirias Research and senior analyst for Moor Insights & Strategy. The author and Liftr Insights may, from time to time, engage in business transactions involving the companies and/or the products mentioned in this post. The author has not made an investment in any company mentioned in this post. The views expressed in this post are solely those of the author and do not represent the views or opinions of any entity with which the author may be affiliated. You can reach him by email at Paul.Teich@LiftrInsights.com.