In a paper published in the journal Nature Physics, DeepMind researchers describe an AI system that can predict the movement of glass molecules as they transition between liquid and solid states. The techniques and trained models, which have been made available in open source, could be used to predict other qualities of interest in glass, DeepMind says.
Beyond glass, the researchers assert the work yields insights into general substance and biological transitions, and that it could lead to advances in industries like manufacturing and medicine. “Machine learning is well placed to investigate the nature of fundamental problems in a range of fields,” a DeepMind spokesperson told VentureBeat. “We will apply some of the learnings and techniques proven and developed through modeling glassy dynamics to other central questions in science, with the aim of revealing new things about the world around us.”
Glassy dynamics
Glass is produced by cooling a mixture of high-temperature melted sand and minerals. It acts like a solid once cooled past its crystallization point, resisting tension from pulling or stretching. But the molecules structurally resemble that of an amorphous liquid at the microscopic level.
Solving glass’ physical mysteries motivated an annual conference by the Simons Foundation, which last year hosted a group of 92 researchers from the U.S., Europe, Japan, Brazil, and India in New York. In the three years since the inaugural meeting, they’ve managed breakthroughs like supercooled liquid simulation algorithms, but they’ve yet to develop a complete description of the glass transition and predictive theory of glass dynamics.
That’s because there are countless unknowns about the nature of the glass formation process, like whether it corresponds to a structural phase transition (akin to water freezing) and why viscosity during cooling increases by a factor of a trillion. It’s well-understood that modeling the glass transition is a worthwhile pursuit — the physics behind it underlie behavior modeling, drug delivery methods, materials science, and food processing. But the complexities involved make it a hard nut to crack.
AI and machine learning
Fortunately, there exist structural markers that help identify and classify phase transitions of matter, and glasses are relatively easy to simulate and input into particle-based models. As it happens, glasses can be modeled as particles interacting via a short-range repulsive potential, and this potential is relational (because only pairs of particles interact) and local (because only nearby particles interact with each other).
The DeepMind team leveraged this to train a graph neural network — a type of AI model that directly operates on a graph, a non-linear data structure consisting of nodes (vertices) and edges (lines or arcs that connect any two nodes) — to predict glassy dynamics. They first created an input graph where the nodes and edges represented particles and interactions between particles, respectively, such that a particle was connected to its neighboring particles within a certain radius. Two encoder models then embedded the labels (i.e., translated them to mathematical objects the AI system could understand). Next, the edge embeddings were iteratively updated, at first based on their previous embeddings and the embeddings of the two nodes to which they were connected.
After all of the graph’s edges were updated in parallel using the same model, another model refreshed the nodes based on the sum of their neighboring edge embeddings and their previous embeddings. This process repeated several times to allow local information to propagate through the graph, after which a decoder model extracted mobilities — measures of how much a particle typically moves — for each particle from the final embeddings of the corresponding node.
Testing the model
The team validated their model by constructing several data sets corresponding to mobilities predictions on different time horizons for different temperatures. After applying graph networks to the simulated 3D glasses, they found that the system “strongly” outperformed both existing physics-inspired baselines and state-of-the-art AI models.
They say that network was “extremely good” on short times and remained “well matched” up to the relaxation time of the glass (which would be up to thousands of years for actual glass), achieving a 96% correlation with the ground truth for short times and a 64% correlation for relaxation time of the glass. In the latter case, that’s an improvement of 40% compared with the previous state of the art.
In a separate experiment, to better understand the graph model, the team explored which factors were important to its success. They measured the sensitivity of the prediction for the central particle when another particle was modified, enabling them to judge how large of an area the network used to extract its prediction. This provided an estimate of the distance over which particles influenced each other in the system.
They report there’s “compelling evidence” that growing spatial correlations are present upon approaching the glass transition, and that the network learned to extract them. “These findings are consistent with a physical picture where a correlation length grows upon approaching the glass transition,” wrote DeepMind in a blog post. “The definition and study of correlation lengths is a cornerstone of the study of phase transition in physics.”
Applications
DeepMind claims the insights gleaned could be useful in predicting the other qualities of glass; as alluded to earlier, the glass transition phenomenon manifests in more than window (silica) glasses. The related jamming transition can be found in ice cream (acolloidal suspension), piles of sand (granular materials), and cell migration during embryonic development, as well as social behaviors such as traffic jams.
Glasses are archetypal of these kinds of complex systems, which operate under constraints where the position of elements inhibits the motion of others. It’s believed that a better understanding of them will have implications across many research areas. For instance, imagine a new type of stable yet dissolvable glass structure that could be used for drug delivery and building renewable polymers.
“Graph networks may not only help us make better predictions for a range of systems,” wrote DeepMind, “but indicate what physical correlates are important for modeling them that machine learning systems might be able to eventually assist researchers in deriving fundamental physical theories, ultimately helping to augment, rather than replace, human understanding.”
Author: Kyle Wiggers.
Source: Venturebeat
