Touched by AI: Competitive intelligence culls new data insights

As artificial intelligence (AI) tooling finds wider use, data-driven approaches to competitive intelligence practices are rapidly gaining traction. As a result, a new generation of decision-makers can probe changing markets and meet rising challenges across various industries.

Change is driven by a deluge of customer data now generated from website activity, surveys and social media. Meanwhile, companies are poised to use the power of new AI tools to continuously monitor market trends and adjust their positioning, offerings and pricing strategies in order to maximize revenue opportunities.

As with so many things today, AI/ML models are seen as a game changer that will help find data insights. The arrival of large language models like GPT presents exciting opportunities for competitive intelligence, according to Kurt Muehmel, who holds the title of everyday AI strategic advisor at AI platform provider Dataiku.

The difficult task of gathering information on competitors and customers can be streamlined via such techniques, he said.

“These models are very good at summarizing and synthesizing text. Therefore, they can be useful to summarize, for example, transcripts of earnings calls, or to flesh out competitive positioning documents if they are provided with accurate data for their input,” said Muehmel.

That is important because, with its wide range of approaches and sources, gathering data as part of competitive intelligence practices can be daunting. Data sources range from industry experts’ blogs or presentations, to financial reports, news media items, public data sources and more.

Increasingly, AI tools, models and processes are essential drivers of competitive advantage, enabling continuous extraction of information that drives strategic decision support.

Modern competitive intelligence algorithms now combine historical and real-time data with machine learning, enabling companies to predict market trends and optimize pricing strategies with remarkable accuracy. This gives organizations a competitive advantage and allows them to respond to changing market trends and consumer preferences in real time.

Businesses can process vast amounts of data to identify patterns and make accurate predictions about future market trends. This information can then be used to make informed decisions, such as product development and marketing strategies, giving companies a much-needed edge in a crowded market.

According to Muehmel, data analytics, AI and automation have made it possible for vendors of all sizes to monitor a broader range of competitors.

“Many SaaS platforms available today enable automated monitoring of competitors’ activities across regions and languages. This is a great benefit, especially for companies that are only beginning to start their competitive intelligence practices,” Muehmel told VentureBeat.

He explained that developing in-house capabilities to build analytics and AI that suits a particular organization’s needs is one of the main ways companies outside the technology space can gain significant advantages.

“Utilizing analytics and AI allows organizations to improve every process in their value chain. Companies that succeed in internalizing advanced analytics and AI capabilities will be the winners in their industries in the coming years,” said Muehmel.

Steps toward a competitive intelligence framework

At the heart of a successful competitive intelligence strategy lies a well-orchestrated cycle encompassing four critical phases: planning and defining the research objectives, gathering relevant data, processing and analyzing the data, and ultimately acting on the insights gained.

Michael Fagan, chief data scientist at enterprise VR company Mesmerise, believes that the most crucial ingredient for any competitive analysis is its data sources, as a single point-of-view dataset can often lead to misinterpreting the output. To overcome this, he suggests utilizing multiple data sources, but warned that each comes with its own biases.

In the course of his industry experience, typical data sources included external markets, social media and website tracking. The first step, of course, is to establish a baseline for understanding. It remains a vital prerequisite for useful AI processing.

“We first needed to align the datasets by understanding the natural distributions and applying weights. This data enabled us to predict the search share pretty accurately on a weekly basis. It also showed our share of the market, what terms and topics were standard and what was up and coming. Having this information initially can be sobering, but this is a baseline,” he said.

“Adding machine learning to the mix further enables you to interpret the recorded patterns and create automated processes so that the intelligence gained is timely enough to take action and positively impact your business over your competitors,” Fagan told VentureBeat. “To stay ahead of the curve, you need to focus on your base data and ensure you have a solid governance structure in place and standard techniques to compensate for biases. Once you have this, you can always be confident that the intelligence layer will add value.”

Likewise, Jo Ramos, distinguished engineer and director at IBM Expert Labs, emphasized the importance of training a competitive intelligence AI model using a large, well-labeled dataset for the specific task it’s designed to tackle.

“AI models require thorough training to accurately capture or represent the patterns in the dataset before it can be applied to real use cases. Today, very few organizations have the skills, software and infrastructure needed to build and innovate with state-of-the-art models like GPT-3,” Ramos said. “The organizations that have pioneered this space have kept many of the enabling tools and technologies proprietary or internal.”

Ramos says that while establishing your competitive intelligence framework, businesses must understand the importance of AI governance –- defining policies and establishing accountability throughout the AI life cycle.

“At IBM, we have an AI Ethics Board that supports a centralized governance, review and decision-making process for IBM ethics policies, practices, communications, research, products and services,” said Ramos. “Doing so helps your models adhere to fairness, explainability, robustness, transparency and privacy principles.”

What’s next for AI-based competitive intelligence?

For his part, Dataiku’s Muehmel said that the most important thing companies can do is to ensure they have a solid strategy for applying analytics and AI to applications throughout their business, including, but not limited to, competitive intelligence.

“Focusing on competitive intelligence, companies should not hesitate to experiment with large language models to see if they can produce relevant suggestions in terms of competitive positioning or if they can otherwise accelerate competitive intelligence data collection and analysis,” he added.

If AI models succeed in gathering information and assisting in visualizing data, teams can make more actionable decisions and save time on information gathering. It is early going for many industries, but some are now on a new path to informed real-time decisions that promise a more competitive edge.