How AI-powered BI tools will redefine enterprise decision-making

Value-creation in business intelligence (BI) has followed a consistent pattern over the last few decades. The ability to democratize and expand the addressable user base of solutions has corresponded to large value increases. Enterprise BI arguably started with highly technical solutions like SAS in the mid-’70s, accessible only to a small fraction of highly specialized employees. The BI world began to open up in the ’90s with the advent of solutions like SAP Business Objects, which created an abstraction layer on top of query language to allow a broader swath of employees to run business intelligence. BI 3.0 came in the last decade, as solutions like Alteryx have provided WYSIWYG interfaces that further expanded both the sophistication and accessibility of BI.

But in many cases, BI still involves analysts writing SQL queries to analyze large data sets so that they can provide intelligence for non-technical executives. While this paradigm for analysis continues to increase, I believe that a new BI paradigm will emerge and grow in importance over the next few years — one in which AI surfaces relevant questions and insights, and even proposes solutions.

This fourth wave of BI will leverage powerful AI advancements to further democratize analytics so that any line of business specialist can supervise more insightful and prescriptive recommendations than ever before.

In this fourth wave, the traditional order of BI will be inverted. The traditional method of BI generally begins with a technical analyst investigating a specific question. For example, an electronics retailer may wonder if a higher diversity of refrigerator models in specific geographies will likely increase sales. The analyst blends relevant data sources (perhaps an inventory management system and a billing system) and investigates whether there is a correlation. Once the analyst has completed the work, they present a conclusion about past behavior. They then create a visualization for business decision makers in a system like a Tableau or Looker, which can be revisited as the data changes.

This investigation method works quite well, assuming the analyst asks the right questions, the number of variables is relatively well-understood and finite, and the future continues to look somewhat similar to the past. However, this paradigm presents several potential challenges in the future as companies continue to accumulate new types of data, business models and distribution channels evolve, and real-time consumer and competitive adjustments cause constant disruptions. Specifically:

1. The amount of data produced today is unfathomably large and accelerating. IDC predicts that worldwide data creation will grow to 163ZB by 2025, up 10x from 2017. With that amount of data, the ability to zero in on the variables that matter is akin to finding a needle in a haystack.
2. Business models and ways of reaching customers are becoming more varied and complex. Multi-modal distribution (digital, D2C, distributor-led, retail, ecommerce), international customers, mobile usage, and marketing channels (social media, search engine, display, television, etc.) have changed the dynamics of decision making and are more complicated than ever before.
3. Customers have more options and can change preferences and abandon brands faster than ever. New competition arises from both tech behemoths like Amazon, Google, Microsoft, and Apple and a record amount of venture-backed startups.

BI 4.0

AI-enabled platforms that will define the fourth wave of BI start by crunching and blending massive amounts of data to find and surface patterns and relevant statistical insights. A data analyst applies judgment to these myriad insights to decide which patterns are truly meaningful or actionable for the business. After digging into areas of interest, the platform suggests potential actions based on correlations that have been seen over a more extended period — again validated by human judgment.

The time is ripe for this methodology to proliferate — AI advancements are coming online in conjunction with the growth of cloud-native vendors like Snowflake. Simultaneously, businesses are increasingly feeling the strain that business complexity and data proliferation are putting on their traditional BI processes.

The data analytics space has spawned some incredible companies capable of tackling this challenge. In the last six months, Snowflake vaulted into the top 10 cloud businesses with a valuation above $70 billion, and Databricks raised $1 billion at a $28 billion valuation. Both of these companies (along with similar offerings from AWS and Google Cloud) are vital enablers for modern data analytics, providing data warehouses where teams can leverage flexible, cloud-based storage and compute for analytics.

Industry verticals such as ecommerce and retail that are under the most strain from the three challenges outlined above are starting to see industry-specific platforms emerge to deliver BI 4.0 capabilities — platforms like Tradeswell, Hypersonix, and Soundcommerce. In the energy and materials sector, platforms like Validere and Verusen are helping to address these challenges by using AI to boost margins of operators.

In addition, broad technology platforms like Outlier, Unsupervised, and Sisu have demonstrated the power to pull exponentially more patterns from a dataset than a human analyst could. These are examples of intuitive BI platforms that are easing the strains, old and new, that data analysts face. And we can expect to see more of them emerging over the next couple of years.

Steve Sloane is a Partner at Menlo Ventures.