How the quest for AI at scale is gaining momentum in the enterprise

Enterprise companies have experimented with artificial intelligence (AI) for years — a pilot here, a use case there. But company leaders have long dreamed of going bigger, better and faster when it comes to AI.

That is, applying AI at scale.

The goals of this quest may vary. Maybe the hope is to boost customer engagement, improve operational efficiencies and unify AI and data workloads. Perhaps the goal is higher growth, more revenue streams and real-time insights.

But the quest for AI Nirvana has never been just about AI. It’s about going beyond harnessing it in specific applications to implementing it at scale, generating value across the organization.

The trend toward AI at scale has gained significant momentum over the past year. Last July, for example, Gartner research analyst Whit Andrews told VentureBeat that the “colossal” AI trend underlying all other AI trends today is the increased scale of artificial intelligence in organizations.

“More and more are entering an era where AI is an aspect of every new project,” he said. That’s because technology tools are better and cheaper, the talent with the right AI skills exists, and it’s easier to get access to the right data, he explained.

According to a January article from Boston Consulting Group, leaders in scaling and generating value from AI do three things better than other companies: They prioritize the highest-impact use cases and scale them quickly to maximize value; they make data and technology accessible across the organization, avoiding siloed and incompatible tech stacks that impede scaling; and they recognize the importance of aligning leadership and the employees who build and use AI.

But the article also maintained that even though scaling use cases is key to generating and sustaining value from AI, most companies do not yet take advantage of the full potential of this approach.

In this special issue from VentureBeat, we’ll be examining the opportunities and the challenges of applying AI at scale and how organizations can get closer to AI Nirvana. It includes a look at how some enterprises are harnessing the power of MLOps to scale AI across the organization, and how experts say organizations can scale AI responsibly. We also take a deep dive into how companies are using synthetic data to boost their efforts to implement AI at scale.

Finally, this issue highlights how several end-user companies were able to launch AI at scale by implementing technology, processes, governance and strategy across the organization.

What does it really mean to apply AI at scale?

Arsalan Tavakoli, SVP of field engineering and a cofounder of data lakehouse platform Databricks, told VentureBeat that applying AI at scale is all about whether AI has become essential to all the company’s business lines.

“It’s whether AI is core to helping you drive new customer experience or product development or operational efficiency,” he said — “[whether] it has become an intrinsic part of your organization’s ability to transform.”

Many Databricks clients, he pointed out, are doing experiments with AI but have no idea how to scale up. Others are farther along, with models in production, but they realize it’s not efficient.

Having the right data with the right technology powering the right models is also essential, said Justin Hotard, executive vice president and general manager for HPE’s HPC and AI business group.

“We’re seeing a much broader interest in AI at scale, not just because of LLMs and generative AI, but because there’s now this recognition of the power and the potential of what you can do with your data if you build the right models,” he said.

Kjell Carlsson, head of data science strategy and evangelism at MLOps platform Domino Data Lab, agrees that figuring out how to make use of more data for ever larger models is certainly part of the AI-at-scale conversation. However, he added that most of the business value comes not from embedding models into applications in individual parts of the business, but from doing that in other parts of the organization.

“You’re going to need to figure out how to do both of those things,” he said.

Where companies are now

The good news is that organizations are maturing in their efforts to implement AI at scale, said Carlsson. The question is, how much and how fast are companies maturing?

The best indicator of AI maturity, he suggested, is the increasing prevalence of chief data analytics officers and other C-suite roles that have an explicit mandate to implement data science and machine learning in their organization. In addition, these executives have control over the data assets that you need in order to be able to execute.

“I think previously there was this massive lack of leadership within the organization, [leadership] that actually was able to take an active role in driving AI-based transformation initiatives,” he said.

The rise of ChatGPT and other generative AI solutions has certainly given companies a kick in the pants over the past few months, added Tavakoli. “I don’t remember the last time I was in a meeting where somebody did not use the word ‘ChatGPT’ in some form or another.”

A year ago, AI and ML were more aspirational for many organizations, he said. “They talked about it, somebody would jokingly say it was great, investors love to hear about it, it’s the way the world is going. But it was tomorrow’s challenge, not today’s.”

Now, he said, leaders are worried about falling behind in an era of fierce competition. “Every CEO’s earnings call is about AI and ML embedded in the business,” he said. “And I’m not just talking about the Netflixes and the Ubers of the world. You’re talking about the Disneys of the world, the banks of the world, the T-Mobiles of the world, the Walmarts of the world — they’re all saying AI and ML is our key to our focus area.”

However, as organizations get deeper into the work, they realize that the most difficult part of implementing AI and ML is not the algorithm.

“It’s all the other stuff behind it,” he said, “like ‘How do I actually figure out how to get good quality data, especially in real time? How do I actually figure out how to develop it and get my data scientists productive, put it in production, iterate on it, and understand when I have data quality issues?’”

One of the biggest challenges, Tavakoli added, is that many organizations felt liberated when they moved their data away from on-premises into the cloud, because they could get “best-of-breed” solutions for everything. But that has led to a “smorgasbord” of tools that all need to be connected.

“What people are realizing is they don’t really have an AI problem, they have a customer-360 problem,” he said. “When they start trying to stitch it all together, it becomes incredibly hard — and then [there’s] dealing with the data and governance around it.”

What companies need to do to scale AI

HPE’s Hotard says that the first thing companies should do to begin applying AI at scale is consider the places where AI can have a positive impact on their business — and whether it is playing offense in the industry, or playing defense (if you don’t do it, someone else will).

Next, if there isn’t already someone in place, appoint someone to lead AI efforts at a senior level. “That’s someone engaged with the C-suite and facilitating these discussions across the business,” he said.

Finally, in terms of AI tools and capabilities, consider enterprise risk and auditability. “It’s going to become important to have the ability to go back and say how you got to the decision,” he said.

The good news is, there are several verticals that have already made significant headway in their quest towards applying AI at scale, said Domino’s Carlsson. “We’ve already hit the tipping point in verticals like pharmaceuticals and insurance, and I would think banking and financial services are there already [too],” he said.

Pain points are still everywhere, he cautioned, from the need to break down technology and data silos to a shortage of high-skilled talent. But today, with the latest technology tools, increased compute and advanced data solutions, the quest for AI at scale can be tackled in powerful new ways that have never been available before.