Anyscale unveils new tool for building and scaling AI and ML development

Scaling the resources needed for machine learning (ML) and artificial intelligence (AI) workloads can be a complex task.

The need to reliably scale development resources for ML training is critically important as organizations are building more models —   many of which never actually make it into production. A Gartner report released in August 2022 found that only 54% of AI models move from the pilot phase into production.

Among the many tools that organizations can use to scale AI and ML projects and potentially get more models into production is with the open-source Ray framework, which hit its 2.0 milestone in August. Anyscale, which is the creator of Ray, goes a step beyond the core open capabilities of the open-source platform with its managed Anyscale Platform and is now extending those features further.

The company’s new release, Anyscale Workspaces, is available today in an early preview. It provides users with a cloud-managed developer environment for building and scaling AI in a repeatable approach.

“We’re trying to get to the point where if developers know how to program on their laptop, and they know Python, that’s all they need to know, in order to do machine learning to build scalable applications,” Robert Nishihara, CEO and cofounder of Anyscale, told VentureBeat.

The key to ML scaling is reproducibility

According to Nishihara, data scientists and developers today are used to designing ML models using Python on local systems.

Anyscale Workspaces extends that same experience developers are used to on their own single systems they are developing on and extends it out to the cloud. Users get the benefit of the Anyscale Platform, which can support hundreds of GPUs and thousands of CPUs.

With Workspaces, Nishihara said that the goal is to help developers iterate faster on models, which ultimately will land in production use cases.

The Workspaces service runs inside of containers in the cloud and is highly customizable. Organizations can specify what AI or ML and data science tools they want installed. Workspaces can also be cloned, making it easy for organizations to reproduce the same development environment for all users.

The reproducibility of the development environment is crucial for rapid iteration, according to Nishihara. With a common, reproducible AI and ML development environment, it becomes easier for developers to collaborate on model development and it also makes it possible for new developers to continue existing projects.

“You don’t want developers to have to spend time fighting with setting up the infrastructure or, or getting it to the way that they ran it previously,” he said. “Now, once somebody has the development environment working the way they like, it can be reused.”

Nishihara explained that prior to Workspaces, interactive AI/ML development on the Anyscale platform, by his own admission, was ‘more painful’. It wasn’t as easy to have a persistent development environment that could easily be customized or reproduced.

Faster cluster setup speeds model iteration

The ability to quickly set up a cluster for training is also being accelerated by Anyscale.

With new updates landing on the Anyscale Platform, the company claims that it can now start  a training cluster up to five time faster than it takes with the regular open-source version of Ray. Nishira said that a complex cluster can be set up in one to two minutes, which enables developers to iterate on models faster.

For example, he said that if it takes 10 minutes for a result to appear because that’s how long it takes for a cluster to startup, then a developer will spend a good amount of time just waiting for things to happen.

“If I submit the job, and it starts running right away, and I get the results, then I  can just do that over and over,” Nishihara said. “So while in theory, users can still do all the same things as they could before, it’s a game changer when it comes to just productivity.”