We are in the great monitoring renaissance

#include <std_disclaimer.h>

Someone told me just yesterday that my head was in the clouds. That I was too much of a dreamer about monitoring, but I really disagree. We are in the great business and application monitoring renaissance!

Today, monitoring systems both open source and from leading vendors are simpler to implement and distill better intelligence about application performance than ever before and better capabilities are coming.

There are a pile of vendors that do all or most of the 5 APM dimensions described by Gartner. The future though is different. It’s something more, something with it’s own intuition to help us normal humans manage things well. And it will be more than a system that helps you become aware and address technical performance issues like today’s APM. It will be a system that helps you manage Customer Experience across all channels.

Someday we may have the internet of things (IoT) because everything will be a sensor, but we already have a lot of sensor data for managing business, applications, networks and platforms.

Many organizations already have sensors that collect performance and availability data from:
– synthetic end-user monitoring
– real user monitoring
– algorithm performance
– transaction tracing
– platform monitoring
– network performance monitoring
– database performance
– visitor analytics
– business performance statistics
– events like product releases

The bigger issue is that much of the above sensor data are still looked at in a non-integrated way.

What organizations need are business analytics and performance systems that give us the traditional shareable KPI dashboards with a layer underneath. That statistically powered, machine learning layer that includes analyzing the streams of “big data” coming from all those sensors in real-time, identifying anomalous behavior and correlating other anomalous events all the way from the technical stack, through to the user experience layer, and ending up with business results.

I was told that this is too complex. That it will never be mainstream.

Yes, performing streaming analysis of data in real-time and correlating that across hundreds or thousands of metrics is complex, and so is a fingerprint sensor on a smartphone. It’s ok if something is complex inside as long as the user interaction is not complex. Well designed products take very complex things and make them simple for users to leverage.

This isn’t anything as futuristic as AI. In fact, to me this seems like the maturation of business intelligence systems applied to customer experience. In the beginning there was the data. The data is big and raw and complex and hard to look at. Over the years we turned that data into information. Delivering reports and dashboards that make it easy to understand and ask questions of the data or build dashboards to show KPIs over time. The fulfillment of BI promise is that software systems can help us turn data into information and into knowledge.

That’s really what we are striving for. That our operational systems are smart enough to self-identify anomalous behavior anywhere is the business / technology stack. Machine detected anomalies effectively create a warrant which needs to be triaged before jumping in to action. But isn’t that what we really want from our business monitoring systems.

Tell me when something unusual is happening and provide all the related things that could be causing it.

Just my 2-cents. It doesn’t seem like rocket science to me.

Ken

Leave a Reply

Your email address will not be published. Required fields are marked *