Bias in AI isn’t an enterprise priority, but it should be, survey warns

A global survey published today finds nearly a third (31%) of respondents consider the social impact of bias in models to be AI’s biggest challenge. This is followed by concerns about the impact AI is likely to have on data privacy (21%). More troubling, only 10% of respondents said their organization has addressed bias in AI, with another 30% planning to do so sometime in the next 12 months.

Conducted by Anaconda, whose platform provides access to curated instances of open source tools for building AI models, the survey of 4,299 individuals includes IT and business professionals, alongside students and academics. It suggests IT organizations are now exercising more influence over AI, with nearly a quarter of respondents (23%) noting data science teams report up through the IT organization. Approval of the AI platforms employed by IT teams ranked third (45%) after performance (60%) and memory (46%).

Respondents said they spend about 39% of their time on data prep and data cleansing, which is more than the time spent on model training, model selection, and deployment combined.

Among respondents responsible for deploying AI models in production environments, the top challenges cited are security (27%), recoding models from Python or R to another programming language (24%), managing dependencies and environments (23%), and recoding models from other languages into Python or R (23%). Python remains the dominant language (63%) employed by data science teams, while a full 87% said they are employing open source software to some degree.

The worst AI myths

The top two biggest data science myths cited are 1) that having access to lots of data leads to greater accuracy (33%) and 2) the perception that data scientists don’t know how to code (31%).

The survey suggests there is also a long way to go in terms of embedding AI within business workflows. Only 39% of respondents said many decisions are based on insights surfaced by their data science efforts. A little over a third (35%) said some decisions are influenced by their work. Only 36% said their organization’s decision-makers are very data literate and understand the stories told by visualizations and models. Just over half (52%) said decision-makers as mostly data literate but need some coaching.

However, the percentage of individuals across an organization that will be employing data science is only going to increase in the months ahead, said Anaconda CEO Peter Wang. “You don’t need to know data science to use data science,” he said.

AI spending drops over 2020

In the short term, however, the survey suggests investment in AI fell somewhat in the last year. More than a third of respondents said they saw a decline in AI investments in the wake of the economic downturn brought on by the COVID-19 pandemic. Only just over a quarter of respondents (26%) said their organization actually increased investments in AI.

Nearly half of respondents (45%) said reduced investments manifested themselves in the form of reduced budgets. Nearly half (47%) said their teams did not grow, while 39% said members of their teams were actually laid off. Just over a third (35%) said projects were put on hold or had their deadlines extended. Just under a third of respondents (32%) said they expect to be looking for a new job in the next 12 months.

There’s no doubt organizations of all sizes are engaging to the best of their ability in what is rapidly becoming an AI arms race. But not all processes lend themselves equally well to AI, so the issue is not just how to build AI models but where best to apply them.