Survey finds talent gap is slowing enterprise AI adoption

AI’s popularity in the enterprise continues to grow, but practices and maturity remain stagnant as organizations run into obstacles while deploying AI systems. O’Reilly’s 2021 AI Adoption in the Enterprise report, which surveyed more than 3,500 business leaders, found that a lack of skilled people and difficulty hiring topped the list of challenges in AI, with 19% of respondents citing it as a “significant” barrier — revealing how persistent the talent gap might be.

The findings agree with a recent KPMG survey that revealed a large number of organizations have increased their investments in AI to the point that executives are now concerned about moving too quickly. Indeed, Deloitte says 62% of respondents to its corporate October 2018 report adopted some form of AI, up from 53% in 2019. But adoption doesn’t always meet with success, as the roughly 25% of companies that have seen half their AI projects fail will tell you.

The O’Reilly report suggests that the second-most significant barrier to AI adoption is a lack of quality data, with 18% of respondents saying their organization is only beginning to realize the importance of high-quality data. Interestingly, participants in Alation’s State of the Data Culture Report said the same, with a clear majority of employees (87%) pegging data quality issues as the reason their organizations failed to successfully implement AI.

The percentage of respondents to O’Reilly’s survey who reported mature practices (26%) — that is, ones with revenue-bearing AI products — was roughly the same as in the last few years. The industry sector with the highest percentage of mature practices was retail, while education had the lowest percentage. Impediments to maturity ran the gamut but largely centered around a lack of institutional knowledge about machine learning modeling and data science (52%), understanding business use cases (49%), and data engineering (42%).

Talent gap

Laments over the AI talent shortage in the U.S. have become a familiar refrain from private industry. According to a report by Chinese technology company Tencent, there are about 300,000 AI professionals worldwide but “millions” of roles available. In 2018, Element AI estimated that of the 22,000 Ph.D.-educated researchers globally working on AI development and research, only 25% are “well-versed enough in the technology to work with teams to take it from research to application.” And a 2019 Gartner survey found that 54% of chief information officers view this skills gap as the biggest challenge facing their organization.

While higher education enrollment in AI-relevant fields like computer science has risen rapidly in recent years, few colleges have been able to meet student demand, due to a lack of staffing. There’s evidence to suggest the number of instructors is failing to keep pace with demand due to private sector poaching. From 2006 to 2014, the proportion of AI publications with a corporate-affiliated author increased from about 0% to 40%, reflecting the growing movement of researchers from academia to corporations.

One curious trend highlighted in the survey was the share of organizations that say they’ve adopted supervised learning (82%) versus more cutting-edge techniques like self-supervised learning. Supervised learning entails training an AI model on a labeled dataset. By contrast, self-supervised learning generates labels from data by exposing relationships between the data’s parts, a step believed to be critical to achieving human-level intelligence.

Spotlight on supervised learning

According to Gartner, supervised learning will remain the type of machine learning organizations leverage most through 2022. That’s because it’s effective in a number of business scenarios, including fraud detection, sales forecasting, and inventory optimization. For example, a model could be fed data from thousands of bank transactions, with each transaction labeled as fraudulent or not, and learn to identify patterns that led to a “fraudulent” or “not fraudulent” output.

“In the past two years, the audience for AI has grown but hasn’t changed much: Roughly the same percentage consider themselves to be part of a ‘mature’ practice; the same industries are represented, and at roughly the same levels; and the geographical distribution of our respondents has changed little,” wrote Mike Loukides, O’Reilly VP of content strategy and the report’s author. “[For example,] relatively few respondents are using version control for data and models … Enterprise AI won’t really have matured until development and operations groups can engage in practices like continuous deployment; until results are repeatable (at least in a statistical sense); and until ethics, safety, privacy, and security are primary rather than secondary concerns.”