Nvidia moves Hopper GPUs for AI into full production

Nvidia announced today that the Nvidia H100 Tensor Core graphics processing unit (GPU) is in full production, with global tech partners planning in October to roll out the first wave of products and services based on the Nvidia Hopper architecture.

Nvidia CEO Jensen Huang made the announcement at Nvidia’s online GTC fall event.

Unveiled in April, H100 is built with 80 billion transistors and has a range of technology breakthroughs. Among them are the powerful new Transformer Engine and an Nvidia NVLink interconnect to accelerate the largest artificial intelligence (AI) models, like advanced recommender systems and large language models, and to drive innovations in such fields as conversational AI and drug discovery.

“Hopper is the new engine of AI factories, processing and refining mountains of data to train models with trillions of parameters that are used to drive advances in language-based AI, robotics, healthcare and life sciences,” said Jensen Huang, founder and CEO of Nvidia, in a statement. “Hopper’s Transformer Engine boosts performance up to an order of magnitude, putting large-scale AI and HPC within reach of companies and researchers.”

[Follow along with VB’s ongoing Nvidia GTC 2022 coverage »]

In addition to Hopper’s architecture and Transformer Engine, several other key innovations power the H100 GPU to deliver the next massive leap in Nvidia’s accelerated compute data center platform, including second-generation Multi-Instance GPU, confidential computing, fourth-generation Nvidia NVLink and DPX Instructions.

“We’re super excited to announce that the Nvidia H100 is now in full production,” said Ian Buck, general manager of accelerated computing at Nvidia, in a press briefing. “We’re ready to take orders for shipment in Q1 (starting in Nvidia’s fiscal year in October). And starting next month, our systems partners from Asus to Supermicro will be starting to ship their H100 systems, starting with the PCIe products and expanding later on this year to the NVLink HDX platforms.”

A five-year license for the Nvidia AI Enterprise software suite is now included with H100 for mainstream servers. This optimizes the development and deployment of AI workflows and ensures organizations have access to the AI frameworks and tools needed to build AI chatbots, recommendation engines, vision AI and more.

Global rollout of Hopper

H100 enables companies to slash costs for deploying AI, delivering the same AI performance with 3.5 times more energy efficiency and three times lower total cost of ownership, while using five times fewer server nodes over the previous generation.

For customers who want to try the new technology immediately, Nvidia announced that H100 on Dell PowerEdge servers is now available on Nvidia LaunchPad, which provides free hands-on labs, giving companies access to the latest hardware and Nvidia AI software.

Customers can also begin ordering Nvidia DGX H100 systems, which include eight H100 GPUs and deliver 32 petaflops of performance at FP8 precision. Nvidia Base Command and Nvidia AI Enterprise software power every DGX system, enabling deployments from a single node to an Nvidia DGX SuperPOD, supporting advanced AI development of large language models and other massive workloads.

H100-powered systems from the world’s leading computer makers are expected to ship in the coming weeks, with over 50 server models in the market by the end of the year and dozens more in the first half of 2023. Partners building systems include Atos, Cisco, Dell Technologies, Fujitsu, GIGABYTE, Hewlett Packard Enterprise, Lenovo and Supermicro.

Additionally, some of the world’s leading higher education and research institutions will be using H100 to power their next-generation supercomputers. Among them are the Barcelona Supercomputing Center, Los Alamos National Lab, Swiss National Supercomputing Centre (CSCS), Texas Advanced Computing Center and the University of Tsukuba.

Compared to the prior A100 generation, Buck said the prior system had 320 A100 systems in a datacenter, but with Hopper a data center would only need 64 H100 systems to match that throughput of the older data center. That’s a 20% reduction in nodes and a huge improvement in energy efficiency.