Azure Stack Finally Takes Microsoft Public Cloud Private
September 22, 2017 Paul Teich
Microsoft announced Azure Stack at its Ignite event in September 2016 and soft-launched Azure Stack at its Inspire event in July, when it announced that the private cloud solution was available for customer orders. The first wave of Microsoft’s Azure Stack system partners – Dell EMC, Hewlett Packard Enterprise, and Lenovo – plan to ship their certified solutions to customers in September. We will be surprised if Microsoft does not announce first customer shipments with those vendors at Microsoft’s Ignite event in late September.
Azure Stack with compete with other hybrid private cloud frameworks, such as OpenStack, Cloud Foundry, HPE’s Project New Hybrid Stack, Oracle Cloud at Customer, and others. While these frameworks offer paths to move applications between customer owned hardware (on premises or off) and public clouds, Azure Stack has a unique relationship with Azure: Azure Stack is mostly the same code base as Azure, implementing the same application programming interfaces (APIs) and the same developer and customer portals as Azure public clouds.
The core differentiating value of Azure Stack private clouds is that applications can be written once for Azure Stack and then run in Microsoft’s Azure public cloud with no changes or special considerations.
Microsoft’s SharePoint, Exchange, and Office 365 products run on Azure and Azure Stack, as do Microsoft’s database, e-commerce, and software development products. Likewise, Azure software partner Bitnami has certified that much of its Azure optimized application catalog will run on Azure Stack, such as Drupal, GitLab, the LAMP stack, Magneto, NGINX stack, Ruby stack, and WordPress.
Timeframes and Target Customers
Microsoft anticipates the first wave of Azure Stack enterprise customers will already be Azure public cloud customers. These Azure customers will have specific use cases for hybrid public/private cloud business models (including data sovereignty, physical security, and occasionally connected use cases), and an interest in integrated systems. These customers are not looking for more cost-effective virtualization or a VMware replacement (“lift and shift”), nor are they general private cloud customers.
The first iteration of Azure Stack will offer compute, storage, and networking services via hyperconverged server hardware, as well as previews of SQL Server, MySQL, and Azure App Service. Hyperconverged servers tightly integrate an X86 compute server, a software-defined storage server, and software-defined networking (SDN) into a single server appliance, typically to support a highly virtualized IT environment like VMware. Most server vendors now sell richly-configured hyperconverged servers.
All hyperconverged servers within a vendor’s Azure Stack stamp, which Microsoft calls a “scale unit”, must have identical configurations. (A stamp is a high-volume cloud hardware purchase configuration and typically includes several racks of servers.)
In addition, the first wave of Azure Stack stamps from Dell EMC, HPE, and Lenovo will have a tight range of Intel Xeon E5 processors and motherboards, networking, and storage features across those vendors’ stamps (see table below). Microsoft wants to ensure that their Azure Stack code base has a solid foundation before they allow their hardware partners to customize their hardware stacks.
Azure Stack Detail By Vendor
Feedback from the first wave of Azure Stack customer deployments in through the first half of 2018 will either reinforce or defuse Microsoft’s hybrid cloud strategy of using the same code base to implement both their Azure public cloud and Azure Stack private cloud products. This Azure software development investment is a big bet for Microsoft – it is the core of Microsoft’s strategy to move enterprise applications out of virtual machines (VMs, mostly running in VMware’s virtualized environment) and into a true cloud framework.
Microsoft’s private cloud solution is based on ecosystem thinking: “Ensure that most ISV applications and services that are certified for Azure will work on Azure Stack.” To achieve this, Microsoft modified the code of their Azure public cloud code base, without interrupting Azure services, so that a subset of Azure will run on a very small scale unit: Azure’s scale unit is 880 servers (it used to be 960 servers), while the first Azure Stack scale unit is four to twelve dual-socket servers.
The minimum cost of an Azure Stack scale unit from one of Microsoft’s partners is about $200,000. Each Azure Stack scale unit will be delivered as a single assembled and tested unit. Dell EMC, HPE, and Lenovo started taking orders for Azure Stack scale units in July, during Microsoft’s Inspire event. Microsoft stated that the first Azure Stack scale units will start to ship in September, in 46 countries.
Initially, Azure Stack will run on a minimum scale unit of four dual-processor Intel Xeon E5 “Broadwell” servers. This is a remarkable achievement for a code base that scales to millions of servers in Microsoft’s Azure public cloud.
Microsoft, Dell EMC, HPE, and Lenovo will spend the rest of this year building their supply chains and ensuring Azure Stack availability and reliability. In addition to the launch partners, Cisco will also start shipping Broadwell-based systems before the end of the year, as well.
In the first quarter of 2018, Huawei Technology will join the growing pantheon of Azure Stack system partners with Intel “Skylake” Xeon SP compute nodes. We predict that all Azure Stack system partners will upgrade their compute nodes to Skylake processors and their “Purley” systems at about that time, after Microsoft and enterprise IT customers have qualified the new processors and systems. Intel launched Xeon Scalable in July; a typical IT qualification cycle for a major processor generational refresh is six months. Leading IT shops are already starting to evaluate Xeon SP chips with plans to start deploying in volume in 2018. Azure Stack will be no different.
Also in 2018, Microsoft will add multi-scale units and multi-region support, new virtual machine types, managed disks, and will deliver Service Fabric and Azure Container Service as a service. Microsoft plans to enable field expansion will be available sometime in 2018, as well, so that customers can upgrade their lower-node-count Azure Stack instances to the maximum of twelve servers.
Deploying and Managing Azure Stack
Azure Stack scale units will be delivered as a preinstalled, integrated whole; a gestalt solution. However, customers will need to manage their Azure Stack installation – both hardware and software. Larger IT shops may rise to the challenge. But as OpenStack deployments already know, there are a lot of moving parts in a private cloud software stack. Azure IT certifications will carry over to Azure stack, as they implement the same code and APIs.
Managing Azure Stack will require more knowledge than configuring a solution in an Azure public cloud, because Microsoft manages its own hardware infrastructure for Azure customers, while Azure Stack customers will manage their own hardware. Will there ever be an Azure Stack management certification? Microsoft has not said yet, but if so it might be modeled after its Azure Certified for IoT program.
Customers may choose to have Azure Stack delivered and operated as a fully managed service, either on premises and off. Microsoft lists several partners who can do so:
- Tieto and Atos use Dell EMC scale units
- Daisy and Cloud28+ use HPE scale units
- Rackspace works with several system vendors
- Avanade, a joint venture between Microsoft and Accenture, works with several vendors
These vendors have been delivering managed public services for Microsoft’s Cloud Solution Provider (CSP) program and are now extending their offerings to include Azure hybrid solutions.
To be clear, every private cloud software stack has or will have this stack complexity challenge. For example, Rackspace is also one of the founders of OpenStack, and has offered fully managed OpenStack private cloud solutions for many years.
Because Microsoft operates Azure, only Microsoft has access to Azure’s administration portal. Azure Stack is managed by a customer or by a managed service provider, and so the only major operational difference between Azure public cloud and Azure Stack private cloud is operator access to Azure Stack’s administration portal. The portal provides operators with workload and health monitoring, capacity management, and patching control, plus access to marketplace services (on a per user basis).
Patching control is one of the more important aspects of Azure Stack licensing. Microsoft will validate Azure/Azure Stack patches with partners (remember that both use the same code base). Operators (customers or managed service providers) will decide when to deploy the patches, but Microsoft’s Azure Stack license will specify a service policy for installing patches. Operators will be expected not to fall too far behind the current patch, but more detail is not yet available, the service policy has not been published.
Underneath the Hood
The top of Azure Stack is the same code as Azure. However, the bottom of the software stack, where software meets hardware, is different. Microsoft implemented a hardware abstraction layer it calls a “Resource Provider” (RP) to customize the hardware underlying the common code base for Azure and Azure Stack.
Azure Resource Manager (ARM) calls on RPs to provide access to physical and virtual hardware services. ARM automates deploying infrastructure, applications, and services (cloud orchestration). RPs are foundational web services for all Azure based infrastructure-as-a-service (IaaS) and platform-as-a-service (PaaS) services.
RPs configure and control underlying physical and virtual hardware resources. Microsoft Azure Stack partners, systems integrators, and managed service providers will use RPs to manage Azure Stack hardware scale units.
There are four foundational RPs: Compute (CRP), Network (NRP), Storage (SRP), and KeyVault (KRP).
- Compute RP (CRP): allows Azure Stack tenants to create their own virtual machines and containers.
- Network RP (NRP): delivers a series of Software Defined Networking (SDN) and Network Function Virtualization (NFV) features for the private cloud.
- Storage RP (SRP): delivers four Azure-consistent storage services: blob, table, queue, and account management.
- KeyVault RP (KRP): offers management and auditing of secrets, such as passwords and certificates.
Microsoft and hardware vendors will write CRPs for differentiated compute resources. CRPs enable Microsoft to operate Azure public cloud using a variety of processors – AMD, Cavium, Intel, and Qualcomm. However, Microsoft is managing end customer qualification of Azure Stack via a staged transition from the first generation of Intel Broadwell and Skylake Xeons in 2018. It is unlikely that Azure Stack suppliers or customers will request ARM-based processors – Cavium or Qualcomm – in 2018. AMD’s Epyc will perhaps be a different story.
We believe there will also be an opportunity for hardware vendors to write CRPs for compute accelerators, if the accelerator has a well-defined instruction set or application programming interface (API). For example, supporting GPUs to accelerate virtual desktop infrastructure (VDI) should be low-hanging fruit. Likewise, for supporting in-memory computing and GPU-accelerated Big Data and machine learning applications.
Operators can upgrade RPs for an existing scale unit to improve performance and Azure Stack scale unit hardware partners can write RPs to enable new and differentiated compute, storage, networking, and security hardware.
The first wave of public partner Azure Stack infrastructure has matured from Microsoft Ignite in September 2016 to Inspire in July 2017
The dual-socket Intel Broadwell-based hyperconverged compute nodes are still there from Microsoft Ignite 2016, with similar processor, memory, storage, and networking options. However, there are differences between vendor stacks.
Management switches and software are now vendor specific. Vendor-controlled system management is table stakes for the vendors, and my guess it is a significant part of Microsoft extending Azure Stack to Technical Preview 3 (TP3). Microsoft uses Technical Previews to preview new software to hardware and software partners and ask for feedback from those partners. TP2 is rare for Microsoft products; TP3 is almost unheard of.
For example, Dell EMC will enable cloud administrators to control service levels, backup policies, and retention choices. In addition, Dell EMC will extend infrastructure management into both platform automation and lifecycle management. Microsoft joined the Cloud Foundry Foundation in January 2017 to support Cloud Foundry in Azure. Dell EMC has stated that Pivotal Cloud Foundry support will extend into Azure Stack.
HPE has partnered with Sogeti (a subsidiary of Cap Gemini S.A.) for Azure Stack cloud native development and application migration services and with Cloud28+ for Azure Stack go-to-market (GTM) campaigns. HPE-Microsoft are opening Azure Stack Innovation Centers in Redmond, Geneva, and via remote mobile units for building proof of concept apps, hosting Azure Stack hackathons, and giving developers access to Azure Stack experts.
Quanta Cloud Technology (QCT) showed the makings of a first-generation Azure Stack (uncertified… certification will be the responsibility of whomever wants to deploy a solution). QCT’s participation speaks to broader supply chain interest in Azure Stack. It also hints that the first wave of OEMs may want to quickly differentiate their stacks in 2018. QCT will present a session about deploying SQL Server on Azure Stack at Ignite.
Perhaps unsurprisingly, Cisco Systems will enter the Azure Stack infrastructure fray in November with differentiated network speed and architecture. Cisco will offer 40 Gb/sec top of rack switching to hyperconverged node network plus integrated in-band management capability. Azure Stack will be managed through a service profile in UCS Manager. We would guess that Cisco had additional certification hurdles courtesy of added NRP software development. Cisco will also add NVMe storage to their Broadwell-based platforms via a host baseband adaptor (HBA), which probably involved some changes to the SRP. The rest of Cisco’s infrastructure falls into line with Dell EMC, HPE, and Lenovo.
Huawei intends to ship Intel Xeon SP processors and optional 25 Gb/sec networking in Q1 2018. The move to Xeon SP will bump DDR4 memory speed from 2.4 GHz to 2.6 GHz and include native support for NVMe storage. Otherwise, there isn’t much different in Huawei’s first Azure Stack configuration from the first wave. However, these seemingly small tweaks involve certifying changes to the CRP, NRP, and SRP.
It is reasonable to expect that the rest of the first wave of vendors will have a response for Intel Xeon Scalable support by the time Huawei delivers its Azure Stack solution in early 2018.
Microsoft, Dell EMC, HPE, Lenovo, Cisco, and Rackspace all offer single-node “proof of concept” (PoC) Azure Stack Development Kit (ASDK) developer systems (either physical hardware or virtual instances), so that software developers can start integrating their apps into Azure Stack’s framework. But these PoC systems cannot be upgraded to a full Azure Stack implementation.
Azure Stack Will Eat Enterprise Private Cloud
Microsoft’s private cloud advantage stems from its decision to write and maintain only one code base for Azure public cloud and Azure Stack private cloud. While Azure Stack will be a subset of Azure functionality, the shared functionality will behave identically on both products. Application developers can choose to scale-in their Azure applications to run in Azure Stack instances or to scale-out applications written for Azure Stack into Microsoft’s Azure public cloud.
Microsoft also has a hardware advantage through its Azure architecture team. We think that Microsoft will eventually have to consider certifying Azure Stack scale units assembled from the same gear its Azure team will start deploying soon – Open Compute Project (OCP) Project Olympus rack-scale platforms. This should be attractive to managed service providers, like Rackspace and Avanade. But key questions for the Azure Stack hardware ecosystem are how fast hardware vendors can carve out defensible, differentiated product niches and how much margin can they generate in the process?
Over 16,000 channel partners attended Microsoft’s Inspire event in July. Every Azure Stack session and vendor presentation at Microsoft’s Inspire channel partner event was packed, with standing room only.
We expect that Ignite will attract the same attention from Microsoft’s massive developer community – last year Ignite had 23,500 attendees. We also anticipate more Azure Stack announcements as the first wave of Azure Stack hardware is delivered to customers in the coming weeks.
The near ubiquity of Microsoft’s enterprise presence – Microsoft owns commanding high-ground in operating systems, development tools, applications, channel partnerships, and developer mindshare – gives Azure Stack a private cloud market potential that OpenStack and other private cloud competitors can only dream of.
Paul Teich is an incorrigible technologist and a principal analyst at TIRIAS Research, covering clouds, data analysis, the Internet of Things and at-scale user experience. He is also a contributor to Forbes/Tech. Teich was previously CTO and senior analyst for Moor Insights & Strategy. For three decade, Teich immersed himself in IT design, development and marketing, including two decades at AMD in product marketing and management roles, finishing as a Marketing Fellow. Paul holds 12 US patents and earned a BSCS from Texas A&M and an MS in Technology Commercialization from the University of Texas McCombs School.