The future of AI is distributed … what that means

Artificial intelligence (AI) and machine learning (ML) are disrupting every industry. Their impacts and integrations will only continue to grow.

And ultimately, the future of AI is in distributed computing, Ion Stoica, cofounder, executive chairman and president of Anyscale, told the audience this week at VenureBeat’s Transform 2022 conference.

Distributed computing allows components of software systems to be shared among multiple computers and run as one system, thus improving efficiency and performance.

But while distributed computing is necessary, writing distributed applications is hard, “it’s even harder than before,” Stoica said.

In particular, distributed computing involved with AI and ML has many challenges – distributed systems vary widely in their difficulty of implementation, and engineers must test for all aspects of network and device failure, as well as for different permutations of failures and bugs.

This has provided opportunity for companies like Anyscale, which offers a set of tools to enable developers to build, deploy and manage distributed apps.

The company was founded by the creators of Ray, the distributed AI open-source framework that simplifies scaling of AI workloads to the cloud. Ray allows users to transform sequentially running Python code into a distributed application with minimal code changes.

Serverless and cloud-agnostic

Anyscale’s option is serverless, cloud-agnostic and supports both stateless and stateful computations. The tool abstracts away servers and clusters and provides autoscaling.

As Stoica noted, depending on the data set, the compute requirements to train state-of-the-art models continues to grow by orders of magnitude. For example, Google’s Pathways Language Model (PaLM) – a single model that generalizes across domains and tasks with high efficiency – has 530 billion parameters. And some of the largest carriers have over 1 trillion parameters.

There is a huge gap between the demands of ML applications and the capabilities of a single processor or server. Similarly, Stoica pointed out that, when Apache Spark was developed and released in 2014, all machines were considered homogenous. But that assumption can no longer be made, as today’s landscape involves many different hardware accelerators.

“There is no other way to support these workloads without distribution – it’s as simple as that,” Stoica said.

There are multiple stages to building an ML application, he said – such as labeling, data-specific training, tuning, reinforcement training. “Each of these stages you need to scale, each of them you typically have a different distributed system,” Stoica said.

Building end-to-end pipelines requires stitching such systems together, then re-managing, re-developing – ultimately what he described as a long and laborious process.

“Our mission is about making distributed computing easier, scaling these workloads easier,” he said of Anyscale and Ray.

AI and ML: Game on

Because AI and ML are so far-reaching, use cases for Ray are “all over the place.” The tool has been used in the financial industry, retail, manufacturing – even in applications for America’s Cup when it comes to training crew members.

Another example is use in game testing. “In online gaming, and online games, you don’t have enough humans in a particular room or area of the game to interact with,” Stoica said.

Addressing the concept of “crawling, walking or running,” Stoica described AI as being essentially where big data was 10 years ago. It’s taking time to mature, he contended, because it’s not only about developing tools, but training experts.

Just a half-dozen or so years ago, for instance, colleges and universities began to provide degrees in data science, he pointed out. Now, more AI courses are being offered, and more and more applied AI courses will emerge, he predicted.

To use a baseball analogy, “we are really in the first inning,” Stoica said.