Intel plots a path to ‘universal AI’ with 4th Gen Xeon Scalable CPU

There has long been a divide between which workloads could run on CPUs vs. GPUs for machine learning (ML) and artificial intelligence (AI) applications. Intel is plotting a path to bridge that divide with its 4th Gen Xeon Scalable CPUs.

ML training has often been seen as the exclusive domain of GPUs and purpose-built accelerator hardware, rather than CPUs. That is a situation that Intel is now looking to disrupt. Intel’s goal is to enable organizations of all sizes to use CPUs for training, as well as for AI inference and data preparation, in a common data center platform that is the same from the edge of the network to the cloud.

“AI is now permeating into every application and every workflow,” Pradeep Dubey, senior fellow at Intel, said during a press briefing. “We want to accelerate this AI infusion into every application by focusing on end-to-end performance of the application.”

To support that vision, today Intel is launching its 4th generation Xeon Scalable processors, code-named Sapphire Rapids. The new processor integrates a host of new capabilities designed to help accelerate AI workloads on Intel’s CPU. Alongside the new silicon update is the launch of Intel’s AI Software Suite, which provides both open source as well as commercial tools to help build, deploy and optimize AI workloads.

Intel Advanced Matrix Extensions (AMX) accelerates AI

One of the core innovations in the 4th generation Xeon Scalable processor, from an AI perspective, is the integration of Intel Advanced Matrix Extensions (AMX). Intel AMX provides CPU acceleration for what is known as dense matrix multiplication, which is central to many deep learning workloads today.

Dubey commented that, currently, many organizations will offload inferencing needs to discrete GPUs in order to meet a desired level of performance and service level agreements. He noted that the Intel AMX can provide a 10x performance increase in AI inference speeds over Intel third generation Xeon processors. The new Intel processor also provides speedups for data preparation as well as training.

“This raises the bar of AI needs that can now be met on the CPU installation itself,” Dubey said.

The path to transfer learning

Much of the hype in the AI space in recent months has been around large language models (LLMs) and generative AI.

According to Intel, the initial training for LLMs will still typically require some form of discrete GPU such as the Intel Max Series GPUs. That said, for more common use cases, where an organization is looking to fine-tune an existing LLM, or retrain an existing model, the Intel AMX capabilities will provide high performance. That’s also an area where Intel is pushing the idea of transfer learning as a primary use case of Intel AMX.

“You can transfer the learnings from your original model to a new dataset so that you’re able to deploy the model faster,” Kavitha Prasad, VP and GM datacenter, AI and cloud execution and strategy, told VentureBeat. “That’s what transfer learning is all about.”

Intel AI Software Suite

Hardware alone is not enough to enable modern AI workloads. Software is also needed.

Intel is now aligning its AI software efforts with the new Intel AI Software Suite, which includes a combination of open-source frameworks, tools and services to help organizations build, train, deploy and optimize AI workloads.

Among the technologies in the AI Software Suite is the Intel Developer Catalog. In a press briefing, Jordan Plawner, senior director of Intel AI products, explained that the catalog provides 55 pretrained deep learning models that customers can just download and run.

The suite also includes SigOpt, which is a technology that Intel acquired in Oct 2020. Plawner said that SigOpt provides tools for hyper-parameter tuning in the ML training stage. OpenVINO, which helps organizations with building and deploying models, is also part of the Intel AI Software Suite.

With the combination of software and hardware that is easily deployed in data center, cloud and edge locations, Intel is optimistic that its 4th Gen Xeon Scalable CPU will help to democratize AI, making it more widely usable and available.

“The issue with AI being in the hands of too few is concerning,” Plawner said. “What’s needed, we believe, is a general-purpose CPU, like the Intel 4th Generation Xeon Scalable processor that can run any code and every workload and enable every developer.”