Red Hat and IBM team up to enhance AIops with an open-source project

AIops is what you get when you combine big data and machine learning to automate IT operations processes, including event correlation, anomaly detection and causality determination. At least, that’s how Gartner defines it.

Based on this definition, as well as coverage of vendors that have products they label with the AIops moniker, you’d be inclined to think that AIops is mostly about anomaly detection and remediation. But what about provisioning, configuration, deployment and orchestration?

These are all essential parts of IT operations that haven’t received as much AIops attention. They also happen to be at the core of Red Hat’s open-source IT automation tool, Ansible. The tool is one of the world’s most popular open-source projects, focusing on using automation to install software, automate daily tasks, provision infrastructure, improve security and compliance, patch systems and share automation across the entire organization.

Now, Red Hat has new plans for Ansible with its other initiative, Project Wisdom.  The company is aiming to take automation to the next level in collaboration with IBM Research. Red Hat refers to Project Wisdom as the first community project to create an intelligent, natural language processing (NLP) capability for Ansible and the IT automation industry.

Tom Anderson, VP and GM for Red Hat’s Ansible Business Unit, said Project Wisdom’s future is what he considers “real AIops, where decisions are being made and content and automation decisions are being created in real-time using AI.” 

The move to focus on AIops is unsurprising, as the sector is growing rather quickly. By 2030, the rise in data volumes and its resulting increase in cloud adoption will have contributed to a projected $644.96 billion global AIops market size.

From Ansible to Project Wisdom

What Ansible does is that it simplifies the provisioning and operations of IT infrastructure by abstracting them into a common language. Instead of having to learn the specifics of each part of this infrastructure — whether it’s Cisco or NetApp, Postgres or MySQL, AWS, Azure or GCP — Ansible offers a layer comprised of modules that talk to the underlying APIs of those infrastructure pieces and abstract them into a common Ansible language.

Ansible’s modules are called playbooks and are written in YAML, a data serialization language that is often used for writing configuration files. Anderson shared that playbooks and roles allow Ansible users to build workflows that interact with the underlying infrastructure and application items. Ansible works, but the problem is that creating playbooks takes significant expertise, which is what Project Wisdom aims to address.

Project Wisdom is “bringing applied AI to automating infrastructure and application deployments, using natural language processing to turn English commands into automation playbooks,” Anderson said. 

The goal is to render automation experts more efficient and help newcomers get up to speed faster. Instead of having to know all the specifics about underlying infrastructure, plus how to use YAML, users can simply type in an English command in natural language.

If you’re expecting something like “Hey Ansible, build my new data center”, however, you may need to manage your expectations. First off, Project Wisdom has just been announced. As Anderson pointed out, the way open-source works is simply by putting software out there for the community to build on. Even though Project Wisdom is out there now, it’s not production quality yet.

Anderson acknowledged that Project Wisdom is not at 100% — more like somewhere between alpha and beta. Project Wisdom will produce syntactically correct YAML, but that may need a little tinkering to get 100% right. You still need to have some knowledge of what you are trying to abstract. 

“Project Wisdom does about 95% of the work for you. You just go in and add whatever credentials you might need and you’re up and running,”  Anderson said. 

However, Ansible covers a wide range of infrastructure items and scenarios, and Project Wisdom won’t work equally well for all of them.

Users can interact with Project Wisdom via a text-only interface for now, even though that’s not really a show-stopper. Getting to voice commands should not be too hard and it’s probably not something most of the target audience would prefer anyway. Despite Ansible’s lineage, we don’t see most system admin types being fond of issuing voice commands to their Ansible.

Project Wisdom was just announced at Ansiblefest 2022 and Anderson said that, “like everything we do at Red Hat, this is an open-source community project. We’re introducing this project to the community and saying, join with us. The more people that are involved in training this model, the better the quality of the output will be”.

There are a couple of points to note here. First, this is not a typical open-source project in the sense of producing software that people can download, tweak and deploy on their own infrastructure. Project Wisdom is powered by an AI foundation model derived from IBM’s AI for Code. As Anderson noted, the requirements for deploying such models are substantial, so this is not the envisioned way to use Project Wisdom.

Red Hat will eventually make a Project Wisdom web service available, powered by IBM Labs and IBM Research. Users will be able to connect to and use this service as part of a plugin for their favorite IDE. Initially, Red Hat is targeting VSCode, but over time Project Wisdom will be made available to more IDEs via plugins. At this point, the goal is to raise awareness and get people involved. Eventually, as the project grows and matures, a commercial service may also emerge out of it.

As Anderson noted, there are many reasons to be cautious about this, including licensing and terms of use. The issue of intellectual property and ownership regarding data points included in datasets used to train AI models as well as model output is emerging across different application areas — from AI text-to-image models to AI coding assistants. Red Hat wants to navigate these uncharted waters carefully.

Project Wisdom’s audience

Previously, IBM had ventured into automating IT anomaly detection and remediation with Watson AIops, but the scope and focus of Project Wisdom is different. Red Hat and IBM Research have been collaborating on Project Wisdom for about a year already. According to Anderson, it was a joint initiative which has developed a good relationship between the two parties.

Initially, the intended audience for Project Wisdom was Ansible users. However, Anderson said the users fall into two communities. The first one is infrastructure IT people, the operations teams and infrastructure owners within IT. According to Anderson, this group has been the bread and butter of IBM and Red Hat for many years.

Secondly, there is the developer community. Many Ansible users — even before Project Wisdom came along — are developers. Developers are also experiencing an explosion in the complexity and they don’t necessarily have all the skills required to both develop and deploy applications in all possible environments.

“We’re just trying to make their job of getting their environment up and running, deploying their application, deploying updates to that application, deploying the infrastructure to run that application easier,” Anderson said. “We use Ansible today to do that. Project Wisdom will make it a lot easier for developers to be able to deploy not just their applications, but the infrastructure that will be required to support those applications that they’re building and deploying.”

The evolution of Project Wisdom

Project Wisdom’s applicability is not just limited to Ansible, Anderson says. The first phase of the project is to convert natural language into syntactically correct Ansible language code in a YAML file.

 Red Hat and IBM Research have outlined the next objective to make it possible to optimize existing Ansible playbooks. In addition to that, the companies hope to do the reverse of what is currently done: Take existing Ansible playbooks and “translate” them from YAML to natural language. To be able to achieve these objectives, the two will need to continue their close collaboration.

IBM Research is behind the AI for Code project, which laid the foundation of the AI models that power Project Wisdom. One of IBM Research’s key goals with this project was to develop foundation models that maintain the highest levels of accuracy possible while relying on a smaller computing footprint. To achieve this goal, AI for Code created a dataset called Project Codenet. It leveraged code created as part of coding competitions and included more context around the code.

As IBM Research notes, Project Wisdom’s models not only meet the state-of-the-art in foundation model technology, like GitHub Copilot and OpenAI Codex, but also exceed the footprint efficiency by 35 times, it claims. The number of parameters has been reduced from 12 billion for the OpenAI Codex and GitHub Copilot to 350 million for Wisdom. At the same time, Project Wisdom’s models exceed the quality of the models both in terms of BLEU score, a widely accepted natural language processing (NLP) metric established by IBM as well as metrics specific to Ansible.

Throughout this collaboration, IBM Research has provided the models and Red Hat has provided the additional data and expertise needed to train and fine tune them for specific Project Wisdom objectives. Anderson explained that Red Hat has a massive repository of playbooks that were used to help train the models. In addition, there is publicly available playbook content on Ansible Galaxy, a community where content is exchanged freely. Red Hat’s own Ansible experts also weighed in on the training and fine-tuning process.

What makes Project Wisdom different

Even though it is not the only coding assistant around, Project Wisdom seems to have a different focus than its counterparts. Contrary to all-around coding tasks, Project Wisdom is aimed specifically at producing infrastructure code. As for the direction IBM and Red Hat are looking to pursue next, the two are already looking at how to refactor complex monolithic application code to the equivalent of 25 new cloud-native microservices.

Given both IBM and Red Hat’s footprint in the enterprise, this is a direction and a partnership that makes sense for them. In addition, according to Amin Vahdat, VP and GM of systems and services infrastructure at Google, more than half of cloud infrastructure decisions will be automated in the next three years. Project Wisdom may be a step in that direction as well.

He also noted that most of the conversation around cloud automation today is about day zero provisioning. There has been a lot of work around the automation of day zero provisioning, but Project Wisdom is going to make that much more tractable across multiple clouds, Anderson added.

The next phase, according to Anderson, will focus on day two operations of these applications and the underlying environment to maintain them: detecting and responding to alerts and events and parameters that are changing in the environment. 

“A lot of those day two operational activities are remediated or changed using Ansible automation,” Anderson said.” I can see a future where the evolution of Project Wisdom may be where systems detect and create a description of a problem and that problem is then automatically turned into a playbook for remediation and no one ever has to touch it.”