IBM Watson opens up AI opportunities for software vendors with embeddable libraries

A decade ago, IBM Watson was the name of a large computing system that made headlines around the world for its ability to compete on the Jeopardy game show.

IBM Watson in 2022 is a very different thing and is no longer a single system or even a single service, but rather is the product brand name for a growing set of IBM artificial intelligence (AI) capabilities. Those capabilities include natural language processing (NLP), speech-to-text and text-to-speech functionality. 

Over the last several years, IBM has been steadily expanding its portfolio of AI services that run as a service on both the IBM cloud as well as on public cloud platforms from Google, Microsoft and Amazon. IBM Watson-powered AI services are also accessible via an application programming interface (API) that enables developers to remotely make use of the AI capabilities within applications.

What had been missing from IBM’s Watson portfolio, however, was the ability to directly integrate core AI services’ code into applications running locally on-premises — or in an organization’s own cloud deployment — without needing to trigger an API call to IBM.

“There is an entire army of independent software vendors (ISVs) and partners who are looking to embed AI into their software products and offerings,” Sriram Raghavan, vice president of AI research at IBM, told VentureBeat. “We have the opportunity to be the provider of enterprise AI core capability that they can use to focus on innovation.”

To help support ISVs and partners, IBM today announced the availability of three new AI software libraries that can be directly embedded into applications. The services include NLP, text to speech and speech to text.

What IBM Watson’s directly-embedded AI software libraries enable

Raghavan explained that the software library form factor adds an additional level of flexibility for software developers looking to make use of AI.

As a software library that is directly embedded into an application, there is no need for the application to connect to an external IBM resource. For applications that need to reside locally or that might be dealing with sensitive information where external calls are not permitted, having code that is part of an application is essential to operations.

Embedding the AI libraries directly into software applications is something that IBM has been doing for its own software for multiple years.

“What ISVs are benefiting from is IBM having to solve problems when we tried to embed it into our own software, as such there is a lot of innovation that has already gone into this form factor,” Raghavan said. 

NLP for example, can potentially be a very large codebase given the complexity and variety of use cases it could serve. With the embeddable software library, IBM has optimized the offering to help ISVs use the code that is needed for the functions they want to offer.

How ISVs can directly embed IBM AI libraries with containers

IBM developed a series of reference architectures for deployments that makes use of its AI software libraries. It’s a model that isn’t all that different from any other modern enterprise application deployment.

The entire offering for NLP can be deployed with three to six containers, running on a Kubernetes container orchestration system, such as IBM Red Hat’s OpenShift platform. IBM is not mandating the use of any specific AI acceleration or inference technology either.

Raghavan explained that, for example, if an ISV wants to use the NLP library to do sentiment analysis over a piece of text, what they are invoking at the time is a pretrained model that is actually doing inference.

“Now, clearly, as part of that, we have done a lot of optimization for some of the smaller models and use cases that will run just fine even on CPUs,” Raghavan said. “There are also reference workflows that as you scale them up with GPUs, you can accelerate performance.”

More to come from IBM Watson in the future

Going a step further, IBM Watson is aiming to make it easier for ISVs to customize the pretrained models without the need to fully retrain the models, which can be a time-consuming and complicated process. Raghavan emphasized that the first three AI software libraries that IBM is releasing are just the beginning, with the potential for additional libraries to come in the months and years ahead.

“We already internally have a richer set of libraries that we use to embed AI into our products,” Raghavan said. “We see this as an opportunity to bring continued innovation from IBM directly to ISVs, the way we bring it to IBM products.”