Collective Intelligence: Where The Promise Of AIOps Is Realized

Application containerization is gaining popularity for organizations as a way to improve their development
speed and efficiency. Containers allow developers to easily pack, ship and run any application as a lightweight,
self-sufficient container that can run virtually anywhere, enabling instant application portability.

In my travels, I speak to a lot of people about artificial intelligence for operations, or AIOps. When I do, I feel I often need to dispel some common misperceptions. Most often, people will immediately latch onto the “A” of the equation, the artificial, and how it can evoke sinister images of automation run amok, the large-scale replacement of internal staff and so on.

Or, folks can focus on the “I,” but in terms of intelligence, that’s isolated. One may think of a data scientist as someone off in some remote lab, crunching numbers and algorithms. Any intelligence gathered gets thrown back to business leadership, who may or may not act on the information.

When I see what leading enterprises are doing in this area, these perceptions completely miss the mark, however. The reality is that significant value is being realized today, and the surface is just starting to be scratched in terms of what’s possible.

People, Process And Technology: Before And After AIOps

To fully appreciate the power of this concept of AIOps, it’s important to start by looking at the history of intelligence within traditional IT organizations. The reality is that IT and operations data sets were siloed as were people, processes, and technologies. Here’s a high-level picture of each:

• People: Historically, people gathered in disparate, isolated groups, with teams focused on networks, applications, storage and so on.

• Processes: Processes were also siloed in nature. When issues arose, for example, processes in place revolved around troubleshooting and remediating the specific technologies in a given administrator’s purview. Across silos, the process, if you could call it that, was to have massive, all-hands-on-deck calls in which different teams shared what they were seeing. From a development standpoint, waterfall-based approaches ruled, where one disparate team, say development, handed a product off to a QA team for testing.

• Technology: Here, too, the technologies employed were narrowly focused on a specific domain. Tools, data sets and data types were distinct. Network engineers worked with time-series data. Application administrators looked at topology-focused views.

There were a number of causes for the continued existence of these silos, but one of the reasons they were most persistent is because of data. In too many organizations, establishing unified, cross-domain intelligence simply didn’t happen.

To me, the real promise of AIOps is realized when it yields collective intelligence that spans traditional silos. Contrast the siloed approaches outlined above with an organization that’s been harnessing the advantages of AIOps:

• People: Roles are filled by generalists. For example, a site reliability engineer will have expertise and insights into a site’s entire computing stack, including coding, applications, servers and networks.

• Processes: Agile, DevOps and continuous delivery pipelines are established, and AIOps-fueled visibility is key to optimizing these approaches. Through this visibility, teams can collaborate seamlessly, iterate continuously and speed up innovation.

• Technology: Instead of isolated tools for specific domains, all teams can gain access to unified, service-centric visibility that spans the entire ecosystem.

Keys To Building An Effective AIOps Foundation

As they set out to establish an AIOps implementation, enterprise IT teams have a number of choices, including whether to leverage commercial offerings or build their own capabilities using open-source technologies. No matter which approach is employed, there are three key pillars upon which a successful AIOps implementation is based:

1. Establish a unified, comprehensive data lake. It’s essential to establish a data lake that ingests and stores a wide range of data sets and data types, including topological data, alarm metrics, log files, configuration management databases and more. These different, disparate data sets need to be normalized and correlated.

2. Leverage the right algorithms. Teams don’t need to reinvent the wheel. The reality is that the algorithms required for AIOps have existed for some time. The key is knowing which algorithm to use at which time.

3. Ask the right questions. Throughout the process, it’s critical to have the right questions in mind and ensure that AIOps delivers the intelligence needed to answer the questions that matter. What’s the optimal mix of cloud and on-premise resources? What workloads should get migrated to a cloud environment? How do issues get identified and preempted before users ever experience a hiccup? AIOps implementations can yield powerful insights into these areas and many more.

The Payoff Of AIOps

The true benefit of AIOps is realized when it delivers truly collective intelligence. This collective intelligence yields invaluable advantages as organizations look to break through legacy silos, fueling true, efficient and meaningful collaboration. In this way, AIOps delivers invaluable insights that fuel optimized operations and service levels. If you think about the insect world, and a swarm of bees or a colony of ants, you can get an analogy of the power of collective intelligence. Consider how massive numbers of individual insects can be so connected, so efficient and so focused on a singular goal. Apply that to the realm of IT operations, and you get a picture of what’s possible.