Have The Public Clouds Killed Hadoop?

The big players in the Hadoop data analytics market continue to hit headwinds in a rapidly evolving market that now includes competition from Amazon Web Services and top-tier major cloud providers that offer enterprises services for managing and analyzing the huge amounts of data that they are generating.

Less than a week ago, MapR Technologies, which along with Cloudera and Hortonworks were the major movers over the past decade in commercializing the open source Hadoop technology, said it is only weeks away from having to shut down operations and laying off more than 120 employees unless the company could nail down more financing or find a buyer.

Now Cloudera, which in January closed its $5.2 billion merger with Hortonworks in hopes of creating a larger and stronger company to compete with AWS, Microsoft Azure, and Google Cloud Platform, is being tossed around in roiling waters after poor quartering earnings, disappointing financial projections for the rest of the year, and the retirement of chief executive officer Tom Reilly were announced this week. All of this news put Cloudera’s stock in an immediate tailspin, dropping more than 40 percent as we go to press.

Cloudera and MapR are both being battered by a changing data analytics space as enterprises opt to manage, store, and analyze their data in the cloud rather than taking on the cost and headaches of doing this on-premises. Companies are also using a slew of other in-memory and accelerated databases and object storage where in the past they might have opted for Hadoop and its HDFS file system. At the same time, Cloudera and MapR are challenged by trying to productize and monetize platforms based on open source technology. Those struggles have shown up in the companies’ financial numbers, which in turn had a ripple effect through the organizations.

MapR founder, president and chief executive officer John Schroeder told employees as well as officials with the state of California and the city of Santa Clara in a May 13 letter that a chain reaction of bad news put the company in a dire position that could result in it shutting down. MapR, which had raised more than $280 million since launching in 2009 and has remained private, was in the process of another company regarding the financing of its debt, which would have eased its financial struggles and allowed it t continue operating into the foreseeable future. The deal was close to being completed – financing was expected in May or June – when MapR’s first-quarter numbers came in.

In his letter, Schroeder called the Q1 results “disappointing” and “unexpected,’ adding that he didn’t entirely understand the reasons for the poor numbers. However, he did say that “they were at least in part due to the sudden, last-minute, and unexpected postponement of several customers’ timelines to making a purchasing decision.”

After those numbers came out, the company MapR was negotiating with on the debt financing pulled out, leaving the vendor in a tight financial situation. Schroeder wrote that unless more investment or a buyer was found by June 14, operations at MapR – whose customers include such large companies as Audi, Cisco Systems, Ericsson, Hewlett Packard Enterprise, and UnitedHealthcare – would cease. He said the company’s board at the time was reviewing letters of intent from two different organizations, though it’s unclear what’s happening with them.

The mixture of customers pulling back on their buying plans and poor first-quarter financial results are having a similar impact on Cloudera, which has 929 customers. The company saw a $35 million operating loss during the quarter and is projecting continuing losses throughout the year. While the quarterly numbers were slightly better than what was expected by financial analysts, the projections for the rest of the year caused worry.

Cloudera was also dogged by other factors that resulted in a slowing of bookings in the quarter by existing customers, which represent more than 90 percent of the company’s usual growth, Reilly said during a conference call with Wall Street analysts yesterday. The merger with Hortonworks “created uncertainty, particularly regarding the combined company roadmap, which we rolled out in March of this year,” he said. “During this period of uncertainty, we saw increased competition from the public cloud vendors.”

In addition, Cloudera in March announced its Cloudera Data Platform, a hybrid cloud and multicloud offering that Reilly said will extend from the datacenter to the cloud and edge that can enable enterprises to better run their artificial intelligence workloads and more easily move workloads from one location to another, whether in the cloud or the datacenter. Among the features will be intelligent migration that will use policy-based controls to automate the movement of data and workloads between on-premises private clouds and cloud object stores and cloud bursting of analytic workloads in public clouds. There also will be adaptive scaling to automatically adjust cloud resources based on demand.

“Large global enterprises have complex business use cases requiring multiple analytic functions,” Reilly said. “These enterprises require hybrid cloud services to have data and workloads in both their datacenters and in the public cloud. What they’re asking for is the ability to seamlessly move their data workloads to the optimal locations to minimize costs and maximize efficiency. They also want one consistent model for security, governance, compliance and management. Finally, they plan to use multiple public clouds and open-source software to avoid vendor lock-in. We believe that CDP uniquely addresses all of these customer requirements.”

A problem is that some customers are holding off on renewing or expanding their contracts until the platform becomes available, first as a hybrid software-as-a-service (SaaS) offering the public cloud this summer and later in the year for private clouds.

How much the new Cloudera Data Platform will help the company in its competition with AWS in particular remains to be seen. Both Cloudera and MapR have pushed to grow beyond simply offering Hadoop platforms in hopes of competing better in an increasingly cloud- and data-centric world. Both have embraced Spark in memory processing, Docker containers, Kubernetes container management, and other emerging technologies in its effort to create more broad platforms for bringing together data from multiple sources. The Cloudera Data Platform is a continuation of the company’s push into the public clouds and into specific workloads such as machine learning.

In the wake of the Hortonworks merger, Cloudera recently has previewed a Kubernetes powered, cloud native machine learning platform – Cloudera Machine Learning – that ensures a unified data access experience across on-premises, public and hybrid cloud environments, and Cloudera Edge Management and Cloudera Flow Management, both aimed at Internet of things (IoT) developers and architects.

However, they’re running up against big, well-financed and well-resourced hyperscalers that have their own solutions. AWS offers a range of services aimed at enabling organizations to manage and analyze huge amounts of data, including the Kinesis Data Stream, Lambda, and DynamoDB, while Azure includes SQL Data Warehouse, Data Factory, Databricks, and Cosmos DB.

Reilly will leave Cloudera July 31. As Cloudera looks for another CEO, board chairman Martin Cole will take over as interim CEO.