Enterprises that are betting the massive amounts of data that they are collecting will deliver the insights they need to make better and faster businesses are finding that they could just as easily be overwhelmed by them. The data is coming at them fast from their core datacenters, cloud infrastructure, and the edge, and the applications that are generating it changing even faster, far outpacing the ability for humans to manage it all.
Automation is not new to the IT sector, but artificial intelligence – and particularly machine learning techniques – are increasingly being leveraged by organizations to keep up with the increasing rate of change, leading to the rise of the new trend – and new buzzword – of AIOps, using AI and machine learning to automate IT operations. (Sorry, you Devs.)
Most enterprises are expected to adopt these new technologies and adapt to the fast-evolving IT landscape with staffs that probably haven’t grown much if at all over the past several years.
“What we see is that IT is facing a tremendous challenge as far as the environment through which they are trying to build to support their applications,” Matt Chotin, senior director of technology strategy at AppDynamics, tells The Next Platform. “They’re getting too large and they are getting too dynamic to manage manually. When you are talking about multiple clouds, thousands and thousands and coming up on millions of containers being started on a weekly basis, IoT [Internet of Things] and the number of devices that you’re going to see at the edge, and then the adoption of Agile releases and the continuous releases where you’re putting new code into all of these environments all the time. It’s just too much for what tends to be a small team of operations folks to manage. IT is being asked to do more and more with budgets that are staying flat, even going down a little bit. You can’t scale your team with humans to work with this complexity and that is where the idea of AIOps is coming from.”
None of this will surprise many in the enterprise space. Data has become the lifeblood of companies and a lot of it is being generated in a highly distributed fashion outside of the traditional datacenter. It’s coming in from the cloud, the network edge and mobile applications as well, and it’s moving between all those points. Collecting all that data, and storing, processing and analyzing it, is critical to businesses in digital world, and IT plays a central role.
“All of this work that you do needs to be done with your customers at the top of mind,” Chotin says. “The point of digital transformation is that software is your business, so you need to make sure that the software that is running is always going to be impacting your business properly and that you do not spend time worrying about minor technology challenges that are not affecting the pieces that are actually influencing your business success.”
AppDynamics and its parent company, Cisco Systems, are rolling out an initiative they say will bring together the traditional world of application performance management and monitoring with AIOps that not only will bring enterprises greater visibility into the myriad applications environments but also predict and remediate problems within the IT environment. Cisco bought AppDynamics for $3.7 billion in 2017 – a day before its scheduled initial public offering – with an eye toward building out its enterprise software portfolio. AppDynamics’ software is designed to monitor and manage a customer’s applications, whether running on-premises or in the cloud. It can proactively find problems before they arise and recommend fixes.
AI and machine learning add to such capabilities, and AppDynamics and Cisco are using them to help create a platform that will enable enterprises to not only monitor and manage their applications but also take in, process, manage an analyze the data from multiple domains – including on-premises, multiple clouds and IoT – detect and remediate issues and work with third-party systems. The platform, which the companies are calling the Central Nervous System (CNS) for IT, focuses on what they are calling the pillars of visibility, insights and actions.
It’s an open platform the companies will expand over the coming months. The first product on the insight front is Cognition Engine, which uses machine learning techniques and AppDynamics’ business transaction data model to manage both applications and infrastructure via diagnostics and automated root-cause analysis, detect anomalies. Cognition Engine is the fruit of AppDynamics’a acquisition in 2017 of Perspica, a company whose stream-based technology applied machine learning to data as it comes in rather than after it’s been stored and whose expertise include using machine learning at scale to process large amounts of data.
The technology uses streaming analytics technology to automatically process and analyze millions of records per second, seeing how metrics correlate and detecting when there’s a correlation problem among datasets. Metrics that deviate from the norm are isolated and the information can be used by third-party orchestration and automation tools.
“When performance degradation is happening, it’s not necessarily because the application has a problem,” JF Huard, chief technology officer of data sciences at AppDynamics, explains. “It could be any other aspects of the infrastructure and being able to put our finger very rapidly there is really where we put our efforts. There’s a lot of R&D efforts and research to commence. The ability to provide recommended actions really tells you how you’re going to fix that problem. But you have to figure out what the root causes, and that’s very key.”
Another product released with CNS is an agent that can gather metrics on all aspects of AWS’ Lambda serverless computing platform.
“The use case that we focus on is where a Lambda function is used as a microservice within a larger application, and that’s where you’re talking about integrating with something that is likely to be business-critical,” Chotin says. “Before people feel comfortable saying, ‘OK this function is going to be included in this architectural change, in some respects it’s going to be included in this application. I want to feel comfortable that I have the same visibility into it as I had using my old model.’ And that’s what this allows you to do.”
Similarly, is integrating its technologies with Cisco’s Application Centric Infrastructure (ACI), which is foundational to the vendor’s intent-based networking strategy. The AppDynamics products also will integrate metrics from ACI. Such integration was expected when Cisco acquired AppDynamics and ties the network in as a critical component to the performance of the application.
“What’s exciting about ACI is that it already took an application-centric approach to how you think about networking and the challenge was that the user of ACI needed to essentially configure the network to say, ‘These are the applications that are running on my network and this is how you identify them and therefore this is how I would move workload or whatever between them,’” Chotin says. “What you get now with the AppDynamics integration is that automatic mapping because AppDynamics already had this understanding of the application and therefore it’s able to look into ACI and say, ‘This is the application that’s running here. Let me identify the network endpoints where the issues might be,’ and it strengthens the conversations between application operations so that they can quickly know where problems why what’s being impacted and move to resolve.”