DeepMind applies AI to controlling nuclear fusion reactors

DeepMind, the AI lab backed by Google parent company Alphabet, today announced that it used AI to successfully control superheated matter inside a nuclear fusion reactor. The lab claims that the system, which is detailed in a paper published in the journal Nature, could allow scientists to investigate how such matter reacts under different conditions.

While DeepMind remains engaged in prestige projects like systems that can beat champions at StarCraft II and Go, the lab has in recent years turned its attention to more practical domains, such as code generation, language processing, weather forecasting, app recommendations, and video compression. DeepMind licenses many of its innovations to other Alphabet-owned businesses, like autonomous car company Waymo and YouTube, and it recently launched a spinoff outfit — Isometric Labs — focused on drug discovery.

“While there is still much work to be done … we are pleased that the results indicate the power of AI to accelerate and assist fusion science, most likely augmenting human expertise in the field and serving as a tool to discover new and creative approaches for [fusion reactor control] and beyond,” Martin Riedmiller, a research scientist at DeepMind, said during a press briefing this week. “[The work] also suggests that there might be potential for wider adoption of deep reinforcement learning on physical systems for complex scientific and industrial machines, from simple motor control to complex robots.”

Nuclear fusion

Nuclear fusion — the reaction that powers stars, including the Sun — promises clean, limitless energy by smashing and fusing hydrogen atoms into helium. Unlike some energy sources, fusion produces no greenhouse gases and only small amounts of radioactive waste. But at the lower pressures possible on Earth, the temperatures to achieve fusion must be very high, typically over 100 million Celsius.

One solution is the tokamak, a doughnut-shaped vacuum surrounded by magnetic coils that can contain a plasma of hydrogen hotter than the core of the Sun. However, the plasmas in these machines are unstable, making sustaining the process required for nuclear fusion a challenge. To ensure the plasma never touches the walls of the tokamak, which would result in heat loss (and possibly damage), a control system needs to coordinate the coils and adjust the voltage on them thousands of times per second.

In search of a solution, DeepMind collaborated with the Swiss Plasma Center at EPFL to develop what the lab says is the first reinforcement learning system to autonomously discover how to control the coils in a tokamak. It could be used to design new kinds of tokamaks and controllers, according to DeepMind, as well as other types of “industrial and scientific” control mechanisms.

“Physics simulations are pretty good at capturing reality for the most part, since we understand the laws of physics a lot better than how people work,” Sam Geen, an astrophysicist at the University of Amsterdam, told VentureBeat via email. “It’s also a very practical use case where the results of the model failing would be quite obvious — your fusion reactor would break … Machine learning can be quite powerful for physics problems where there’s a clear link between input and output, but where what happens in-between is reasonably predictable. One thing that machine learning can be good at is taking a huge set of complicated inputs, like the controls of [a] reactor, and mapping out what the result will be very quickly.”

Reinforcement learning

Reinforcement learning gives an AI system a set of actions it can apply to its environment to obtain rewards or reach a certain goal. The system — which usually starts by knowing nothing about the environment and selecting random actions — receives rewards based on how its actions bring it closer to its goal and eventually learn sequences of actions that can maximize the rewards.

Reinforcement learning is leveraged heavily in robotics and other types of industrial applications, such as driverless cars, control simulations, and content recommendations. The approach has also helped researchers master complicated games such as Gran Turismo, StarCraft 2, and DOTA 2, which weren’t solvable to the human expert level with previous AI techniques.

DeepMind’s reactor-controlling system trained using simulation tools built by researchers at EPFL. In fusion research, simulation tools are necessary because the reactors currently in operation can only sustain the plasma in a single experiment for up to a few seconds, after which they need time to reset.

DeepMind’s system learned to control a tokamak — specifically the TCV Tokamak at the Swiss Plamsa Center — in simulation over the course of many hours and then transferred those skills to the real TCV. It’s an impressive feat, considering that existing plasma-control systems require separate controllers for each of TCV’s 19 magnetic coils. While traditional controllers use algorithms to estimate the properties of the plasma in real time and adjust the magnet voltages accordingly, DeepMind’s system uses a single algorithm to control all of the coils at once, automatically learning which voltages are best to achieve a “plasma configuration” directly from sensors.

As a demonstration, DeepMind first showed that the system could manipulate aspects of the plasma with a single controller. The lab then used the system to create a range of plasma shapes being studied by plasma physicists for their usefulness in generating energy. For example, it made a “snowflake” shape with many “legs” that could help reduce the cost of cooling by spreading the exhaust energy to different contact points on the tokamak walls. And DeepMind stabilized two plasmas inside the tokamak simultaneously — a first for TCV.

“This is one of the most challenging applications of reinforcement learning to a real-world system so far,” Riedmiller said. “But, to be clear, that does not mean we have solved the fusion problem. What it does represent … is a significant significant step in our understanding of how we might design new flexible tokamak controllers that now allow us to specify the ‘what’ rather than engineering the ‘how’ — the ‘how’ is now learned by the AI.”

Challenges ahead

DeepMind’s achievement, while significant, is only one step toward a viable source of fusion energy. As the lab notes, simulating a tokamak requires many hours of computer time for one second of real time. The condition of a tokamak can change from day to day, moreover, requiring algorithmic improvements to be developed both physically and in simulation.

“[The DeepMind paper does] bring up the one question I had in mind, which was this issue of ‘nonlinear dynamics,’” Geen said. “Most of what we learn in school is ‘linear’ — if you change the inputs slightly, the outputs change slightly as well. However, a lot of real-world physics is ‘nonlinear’ — if you change the inputs slightly, the outputs change a lot over time. This is the ‘butterfly effect,’ and it comes up in problems with fluids like weather and climate, interstellar gas, and the plasma in fusion reactors, where unpredictable things can happen. Their bet is that the butterfly effect isn’t a big deal if they control things carefully — sometimes properties of fluids are predictable, like if you turn on the shower you know the flow rate of water coming out will stay roughly the same over time, even if you don’t know where every droplet will end up.”

There’s also uncertainty about when fusion power will be ready for commercialization. Estimates range from 20 to 30 years, which doesn’t include scaling up — a potentially decades-long subsequent process.

Still, DeepMind asserts that AI could help to accelerate fusion energy’s path to market.

“Few of today’s major problems in science can be reduced to a small set of elegant or compact formulas to be solved by one person or one small team,” DeepMind’s Jonas Buchli said during the briefing. “We believe that AI is the multiplier of human ingenuity, unlocking new areas of inquiry and enabling us to reach full potential. [T]oday, AI systems are becoming powerful enough to be applied to many real-world problems, including scientific discovery itself.”