Cross Data Streams to Release the True Power of Your AIOps Tools

Regardless of where you are in your AIOps journey, chances are you’re already trying to take your AIOps tools to the next level of efficiency. After all, optimization and efficiency are two primary values of adopting AIOps. The quickest way to get more from the tooling you’ve put in place is to include an analysis engine that uses unsupervised machine learning to combine all your data streams – including any non-technical data you may have.

Artificial Intelligence in Observability and Service Management

It takes just a moment of web searching to find Artificial Intelligence (AI) and Machine Learning (ML) as a descriptor of a myriad of business tools, especially in IT Management, from security management to application observability. We get asked why a more general AIOps platform is needed when most specialty tools have some level of ML or AI assistance.

Before we get to the answer, here’s a quick AI refresher:
– Not everything labeled ML can actually learn
– Not everything tagged as AI is actually intelligent

There are many different ways to deliver “AI” as part of a solution, and most of them aren’t applying intelligence at all. Keep that in mind as we move forward in our discussion.

Why do I Need a Standalone AIOps Tool / Platform?

A common misconception is to believe that if you’re running a set of point AIOps tools, that you’re reaping the benefits of AIOps. While you’re certainly better off than not running automated and/or intelligent tools, point tools are still point tools. This Observability vs. AIOps blog post digs into concepts like anomaly prioritization and criticality, but that’s just one small example.

Even so, SREs, DevOps and DevSecOps tend to specialize their work, their tasks and their goals, such as  application performance, infrastructure efficiency and intrusion detection. Each of their specific spheres of influence centers around the data they get from their central tool (application performance, end user experience, IP logs, etc.).

No matter how good or how SMART a specialty AIOps tool like Observability or Security Management is, each tool operates in its own silo. Ironically, some tools talk about breaking down silos, but it is difficult for them to do so within their contained domain. They do not have a holistic view of how systems (outside of their specificity) work together.

What’s left are broad swaths of operations, data, processes, incidents and more – that do not tell a cohesive story. These parts are still in silos – in some cases MUCH bigger ones. Depending on which tools you own, these silos can shrink or expand.

Crossing the [Data]Streams

In Ghostbusters, we learned that Crossing the Streams was bad. Right up until crossing the streams was the solution to the bigger problem.

It’s the same with your company’s AIOps Data Streams.

Notice I said “Company”, not “IT Organization” – also notice that   “AIOps” is crossed out.

At any given moment, an organization is creating, processing, storing, and analyzing  information from a plethora of sources including IT operations, finance,  fulfillment, sales and marketing. Within IT, there can be more granular data – infrastructure, cloud, APM, Observability, Security, Logs, Digital Experience, Google Analytics, etc..

And across all of them is a blanket box: the Unified Intelligence Engine.

 

Don’t Cross the Streams Without Unsupervised Machine Learning

No, it won’t create “positronic reversal” – but you can’t just feed multiple data streams into any analytics engine. If your machine learning requires supervision, it won’t break through the silos, even though you’re feeding it more data.

What is Supervised Machine Learning?
Supervised machine learning is when you establish a baseline of rules based on a known set of data that your AI system understands. Put another way, it means that the Observability ML engines know that their ingested data is about latency, load, performance, errors, end users, etc.They also recognize that there is a tech stack and a distributed service map that must be considered when making decisions.

In other words, your observability ML won’t be of any assistance when trying to answer whether sales are down the first three days of the holiday season, nor WHY they’re down. Similarly, marketing analysis tools won’t identify your new cloud servers that are causing a 30% longer response time from a year ago (honestly, they probably won’t even KNOW that response time is slower).

Supervised ML is great for point management tools (Observability, security, etc.), but it keeps these data in their silos, and is therefore unable to analyze across the whole system.

What is Unsupervised Machine Learning?
In contrast, unsupervised machine learning doesn’t require pre-knowledge of the meaning of data. It simply analyzes data streams, identifies patterns and anomalies, breaking down all the different data streams to find causality.

Realizing the Full Power of AIOps Tools
The classic IT management tool analogy is to talk about identifying that problem that is super hard to understand – the needle in the haystack. But is that really the problem? No! The real problem to discover is how the needle got put there in the first place.

When you have no artificial (LOL – no pun intended) limits to what you’re analyzing, you can find some amazing correlated and causal results . Not only will you be able to bust through technical management silos, the whole world of business analytics can be your oyster.

Example: figuring out that your finance team changing the limit for individual purchasing decisions from $500 to $250 would result in 20% lower retail sales – because the performance of a few specific transactions is suffering from slightly older infrastructure that would normally be replaced adhoc by empowered infrastructure operators.

Maybe you’ll even find that the best performing updates happen on eclair day in the cafeteria. REALLY – ANYTHING is possible!