The long-tail of AI problems requires hyper-customized solutions, not a silver-bullet

This article was written by Dr. Roey Mechrez, CTO, BeyondMinds.

One of the fundamental elements of product development is solving a problem that many people have. Successful products address many local problems of scattered individuals in a unified, easily repetitive way. Think of products that enable people to conduct online meetings: as the world around us changed over the last year, face-to-face meeting shifted into screen-to-screen, camera-to-camera. A product that enables instant multi-party video calls to anyone and anywhere — as long as they have an internet connection — is indeed a great solution. In theory, AI products should follow the same principle: a repetitive solution that meets common needs shared by all users, with a set of features that are similarly used by all users.

Unfortunately, when it comes to AI things are far more complicated. Generally speaking, AI solutions create data-driven predictions to solve pre-defined problems. These problems are as diverse and widespread as the businesses that order them, across industries, markets, and business cases. Even two companies competing on the exact same market share with similar offerings typically require very different AI solutions: these two seemingly similar companies have different data, different pain points and different business objectives that AI can help solve. To do so, these AI solutions have to be hyper-customized and tailored to these needs. With AI, there really is no “one-size-fits-all.” That specificity characteristic is one of the core challenges in implementing AI at scale nowadays.

The ‘specificity’ challenge of AI

Let’s dive into what it means to create a hyper-customized AI solution. Four key factors make AI problems so diverse, that no silver-bullet solution can address all these problems:

1. Data. There is no AI without data. It can even be said that AI is a fancy way to solve data-related problems. That is, a software that uses data to come up with a recommendation or a prediction regarding this data. A company’s data is one of its precious assets, it’s considered highly sensitive, and it changes greatly from one company to another. Think of customers’ claims in the insurance business: most insurance companies deal with the same process of assessing customer insurance claims, and they all share a common pain point (reducing manual processing which is slow, expensive and prone to human error). But despite these similarities, no single AI solution can solve the claims automation process for all insurance companies. That’s mainly because of the great variation in the data of each of these companies: they each have their own specific data, coming from different distributions, organized and sorted differently, framed in different fields, and impacted by different noise factors and other dynamics.
2. Requirements. Two companies facing the exact same problem can decide to approach this challenge in a very different way. Take CRM for example. All businesses sell products to customers, and most enterprises use some type of CRM to store customer information, keep track of prospects in the pipeline, and nurture them until they convert into paying customers. It sounds like a very repetitive and universal problem, yet if you ever worked with a CRM (such as Salesforce), you probably know that there’s a significant level of customization between companies according to their needs and requirements. As a result, this CRM tool looks different in each company. One of the reasons that Salesforce is such a great product is that on top of its core capabilities it can be customized to address each business’s specific requirements. From a development perspective, enabling this customization is a serious challenge.
3.  Needs. While one business may need a solution that automates one step out in its internal process, another company might want to automate another step. Some companies look for a fully automatic solution, while others need to keep a human in the loop to make the final call. Take fraud detection in the financial services world as an example. The high-level need is similar across all companies — monitoring transactions and flagging those suspected as being fraudulent. Yet, in reality, this process is complex, and banks rely on a range of tools, employees, teams and experts to combat fraud, and face regulation that varies between states and countries. Bottom line: these FIs share a common goal, but have very different needs to help them reaching it.
4. Constraints. On top of these external challenges, customers that wish to implement AI face their own constraints in the process — which are specific and different to each company. These constraints can be the need to add ability to explain to the AI solution, special security constraints on the data, the ability to collect feedback from users to fine-tune the AI model, and keeping the solution fair and ethical. For example, using super-sensitive image identification hardware for detecting manufacturing defects could be beneficial in an airline assembly line — but doesn’t make sense in a textile factory.

Spending the last decade on AI research and implementation, my observation is that these challenges are inherently specific, differing drastically across companies — even when the AI application is identical. This phenomenon was also termed “the long tail problem of AI.” In my view, as much as 80% of AI solutions are so specific, that they cannot be solved with a vertical product that uses a repetitive, cookie-cutter approach.

This brings us to the “buy vs. build” dilemma. With AI, oftentimes buying a solution is not even an option, since enterprise problems that AI can potentially help solve are so specific, shaped by the particular data, constraints, requirements and needs of each enterprise. This realization pushes many organizations to try and build their own internal AI center of excellence — a huge (and expensive) feat for any company, with Fortune 1000 companies spending over $50M annually on AI adoption. More often than not, these companies realize that developing an AI solution from scratch for each use case is a painfully slow and expensive process, prone to many “first time” errors.

But there’s another way. A new type of solution that on one hand is a viable, robust AI product, which at the same time can be fully customized to address the “long tail of specificity.” Such a product will need to be agile enough to make these customizations quickly, addressing the core challenges mentioned above. As the AI landscape becomes increasingly complex — and crowded — the struggle between solution providers intensifies over a solution that combines AI model robustness with the flexibility to customize AI solutions to each customer. No doubt, these are fascinating times to be in the AI domain and see how this story unfolds.

Roey is the CTO and a Co-founder of BeyondMinds, a start-up that helps enterprises achieve sustainable value from AI. The company develops a self-adapting AI platform that provides the building blocks for creating resilient AI solutions, that withstand real-world production environments.