Microsoft is ramping up its Azure public cloud in its effort to gnaw away at the hindquarters of Amazon Web Services, which it trails in the ongoing drive to herd enterprise computing into the cloud. Data services figure prominently in Microsoft’s plans, just as they have at rival AWS.
“We aspire to build an intelligent cloud back-end for the world’s applications running on all devices,” Microsoft CEO Satya Nadella told the keynote audience at his company’s Build 2015 developers conference in San Francisco. Whatever your opinion of Microsoft’s CEO, now into his second year of the post-Ballmer era, you certainly can’t fault him for understating his vision for Azure.
Among the keys to Microsoft’s plans for cloudy domination are four Azure services rolled out this week: Azure SQL Database elastic database pool, SQL Data Warehouse, Data Lake unstructured data repository, and Service Fabric for microservices – a boon to those cloudy developers who are jumping on the containerization bandwagon.
SQL Database Flexibility
The old familiar Azure SQL Database has learned a new trick, called the elastic database pool, which operates pretty much exactly as its name implies: elastically.
The elastic database pool was introduced during the keynote by Scott Guthrie, Microsoft executive vice president for cloud and enterprise, who said that the pool is designed to allow companies to “better manage lots of databases at even lower cost, and is perfect for SaaS solutions.”
According to Guthrie, a core challenge that SaaS developers have when architecting large database applications for their customers is choosing and creating an appropriate data model. They could use one large, shared database with lots of customers sharing the same data tables, but even though this method has advantages in terms of resource costs, it introduces high operational complexity into applications, he explained. Another tactic is to maintain a separate database for each customer, which is great for data isolation and separation but not so great in terms of costs, since each database requires its own resources.
The new elastic database pool service, Guthrie says, “gives you the best of both worlds.” The pool allows you to isolate each customers’ database, but to serve them from a single resource pool while Azure dynamically manages each database’s requirements. “This allows you to smooth out the peaks and dips of different databases within your SaaS app, and allows you to end up using far fewer resources when running your application,” he said.
The bottom line? Your bottom line: “It saves you lots of money, while also improving the overall operational complexity of running your SaaS solution,” he said.
Part of that reduction in complexity is the ability to make changes to a pool of databases simultaneously, a feature that attracted the attention of Al Hilwa, program director for IDC’s application development software research, with whom The Next Platform spoke after the database announcements.
“You can change a schema on one database and it will propagate throughout automatically,” Hilwa told us. “All the glue for that will be handled, all the logging will be handled, so you don’t have to write your own scripts and manage your own system, and that’s super-important if you’re running hundreds of databases.” From his point of view, these new capabilities provide not only cost benefits, they also enhance agility, productivity, and development speed in addition to a quicker rollout of changes.
But Hilwa cautioned against drinking too deeply from this particular cup of Azure-flavored Kool-Aid. “You still have to do a lot of testing, making sure that what you’re rolling out is actually going to work,” he said. “You probably should try it on smaller subsets of your machines; it’s still not one button that you want to press, like they say in the demo.”
Data Warehousing For Analytics
If elastic database pools give developers more elasticity when serving up database services, Azure’s new SQL Data Warehouse provides more flexibility and capability when analyzing data, according to Guthrie.
“We’re in a unique world right now in terms of the expectations people have in writing intelligent apps, and companies embracing analytics will be much more competitive in this disruptive economy than those who don’t,” he said. Guthrie has a point. It’s a dog-eat-dog world out there in e-commerce land, and the dog that digs up the most succulent bones is not necessarily the biggest, strongest, or fastest pooch. It’s the one that knows where those bones are buried, and data analytics is an info-bone dowser.
Not that data analytics is a simple matter of checking sales records; it requires amassing appropriate mountains of data, and then unleashing analytics software upon it. “In the past,” Guthrie said, “data warehouses took months to set up, and integrating analytic systems was hard.” Azure’s SQL Data Warehouse aims to take some of the pain out of that chore by providing a suite of analytic services with the intent of simplifying the process of developing systems that can answer business-related analytical questions.
Azure SQL Data Warehouse, as explained in a Microsoft blog post by Microsoft vice president of data platform, cloud, and enterprise T.K. Ranga Rengarajan, is “the first enterprise-class cloud data warehouse that can dynamically grow, shrink and pause compute in seconds independent of storage, enabling you to pay for the query performance you need, when you need it.”
Specifically, Rengarajan explains: “SQL Data Warehouse is based on the massively parallel processing architecture currently available in both SQL Server and the Analytics Platform System appliance, and will work with existing data tools including Power BI, for data visualization, Azure Machine Learning for advanced analytics, Azure Data Factory for data orchestration, and Azure HDInsight, our 100 percent Apache Hadoop managed big data service.”
Guthrie promised that any of SQL Data Warehouse’s components can be set up and provisioned “in minutes” using the new SQL Database Service, and that the new SQL Data Warehouse Service will dynamically manage petabytes of data and dynamically scale compute resources to munch on that data and spit out helpful analytics.
Don’t expect to get aboard the Data Warehouse with your elephantine queries immediately, however: a preview of it won’t be available until later this year.
Exabytes Of Data In Petabyte-Sized Files
Not all data, however, is as easily manipulated as that stored in relational SQL databases – or, for SQL Data Warehouse’s purposes, in NoSQL, Hadoop, or other sources such as SAP – so to help companies manage and analyze structured or unstructured data such as from the data firehoses promised by the Internet of Things, Microsoft has introduced the Azure Data Lake Service.
“This new Data Lake Service that we’re announcing today is really a revolutionary way to store and process data,” Guthrie humbly intoned. He told his audience that the service was built after over a decade of “real-world learnings” by Microsoft’s own services managing exabytes of data. In perhaps a wee bit of overstatement – or, at minimum, an imperfect understanding of English definitions – Guthrie claimed that Data Lake can store “literally an infinite amount of data,” although with the capacity of exabytes of data, including individual files of up to a petabyte in size, its storage capacity may not be literally infinite, but it ain’t shabby.
After filling a data lake with bits and bytes in essentially any non-relational structure, all in its original form, you can run analytics or process the data to your heart’s content, at any scale. “You don’t have to worry about setting up or managing your own clusters to do this,” Guthrie said. “Instead you can focus on the task at hand that you want to accomplish, and the service will automatically scale out for you and apply the appropriate resources to get it done.”
As might be expected, the Data Lake service has what Guthrie described as “enterprise-level” security to protect all that data from prying eyes. It also exposes data using the standard Hadoop HDFS API, so the learning curve won’t be too steep for Hadoopians interested in dipping their toes into the Lake, seeing as how existing analytical software that takes advantage of HDFS will be ready to go.
Is Microservices Just A Buzzword?
All this week’s talk of SQL, analytics, and Hadoop capabilities was all well and good, but no cloudy discussion – or product rollout – would be complete without delving into the hot topic of containerization, and with it another buzzword du jour, microservices. And Mark Russinovich, Microsoft Azure’s CTO, was more than happy to oblige during a technical session discussing what he referred to as the next-generation of Azure.
Russinovich offered up a laundry list of next-gen Azure capabilities – notably an upgraded Resource Manager family for better handling of lifecycle management, access control, resource communication, and other resource-related matters. He also announced Version 2 of Resource Manager’s network storage and networking APIs that allow asynchronous and parallel operations – just some of the improvements over Version 1, which Russinovich described as having “clunky network modeling.”
But what caught not only our attention but also the attention of IDC’s Hilwa was Azure’s new Service Fabric, which eases the advance of microservice-based applications, and thus, of containers. (The Next Platform detailed Microsoft’s Nano Server implementation of Windows Server and its support of containers earlier this month.)
“What is a microservice-based application?” Russinovich asked – then answered. “A microservice-based application is one that you decompose into tiny parts.” All well and good, according to Hilwa, but “microservices” remains a bit of a squishy term, one that a lot of cloudy vendors are tossing about these days. “It’s like ‘the new XML’,” he said. “It’s ‘the new cloud’. Or whatever. It’s a buzzword.”
Russinovich, however, believes that there’s a lot to be said for truly independent and well-integrated microservices, seeing as how they provide developers the ability to run and test each part in isolation. Also, if you set up strong and clear communication pathways among microservices, it not only simplifies adding new microservices, easing the addition or subtraction of features and feature-updates in a decentralized manner, but also allows for easier application-scaling by adding new microservices.
Despite the marketing haze that Hilwa described, Russinovich stands stalwartly behind Microsoft’s position on microservices, and announced that Redmond is releasing an Azure Service Fabric that greatly simplifies the integration of microservices into containerized applications. And Hilwa told The Next Platform that they definitely have their role in the creation of new applications bound for containers. What’s real about microservices, he said, is that they’re a new way of designing and architecting software. “The people who have gotten – so far – the most miles out of using it are the container people,” he said. “They’re saying, ‘Look, if you build with containers, then you’re building microservices.'”
Containerization, Hilwa said, is still in its early days, but it’s gaining adherents. “People are certainly using them in the startup scene, people like Groupon or Netflix – all these guys. They’re big on containers; they’ve already sort of re-architected around them. A lot of the ‘New Economy’ people are using containers.” What’s less certain, he said, is how long it will take the use of containers to move into more-established “Old Economy” players such as insurance, banking, or hospitals. “It’s going to take some time,” he believes.
The reason for that slow growth is simple: re-architecting major software installations is simply not worth the time and expense – a slow process that may take a decade or two during the usual app-upgrade cycle. “We still have apps being run in critical places that were written thirty years ago – in COBOL,” Hilwa said.
“In most cases it’s about new apps,” he said. “In some cases it’s about moving existing apps to the cloud, but in most cases it’s about writing apps from the get-go that are designed to work in a cloud environment.”
Hilwa maintains that if you’re a new company, using a microservices-based architecture in a containerized cloud environment is a no-brainer. “If you have the luxury of starting from scratch, it’s the only way you do things.” On the other hand, older companies will likely run a blend of architectures until the cloudy crossover is complete. “It’s a hybrid world,” he said.
Microsoft: ‘It’s a Cloud, Cloud, Cloud, Cloud World’
It may be a hybrid world here in 2015, but Satya Nadella’s aspiration to “build an intelligent cloud backend for the world’s applications running on all devices,” along with Microsoft’s investment in beefing up its Azure services and infrastructure, reveals an increasingly cloudy forecast for the skies above Redmond – as well as above the 18 compute regions containing the over one million servers that Microsoft says comprise its Azure offering.
But for Microsoft to catch AWS, it has its work cut out for it. Just last week, for example, data from the Synergy Research Group, showed that AWS held a 28 percent cloud infrastructure services market share. Microsoft was in second place, but far behind with only a 10 percent market share – a nearly three-to-one margin that Hilwa said “sounds about right.”
To be sure, the rapidly expanding cloud industry is about more than just infrastructure, and Wednesday’s announcements indicate that Microsoft is hitching its wagon to not only infrastructure, but is using its 40 years of software chops to develop a broad and expanding range of services. “I think that the database announcements are interesting because it shows that they’re thinking cloud-first,” Hilwa believes. “They’re finally putting in the R&D.”
In point of fact, however, it may not be all that important who’s now leading in the race to move commerce, communication, entertainment, analytics, energy management, medicine, consumerism, your thermostat, and more to the cloud. That pie is growing so rapidly that even a small slice can be mighty filling.