Hydrogels can learn to play Pong

Pong will always hold a special place in the history of gaming as one of the earliest arcade video games. Introduced in 1972, it was a table tennis game featuring very simple graphics and gameplay. In fact, it's simple enough that even non-living materials known as hydrogels can "learn" to play the game by "remembering" previous patterns of electrical stimulation, according to a new paper published in the journal Cell Reports Physical Science.

"Our research shows that even very simple materials can exhibit complex, adaptive behaviors typically associated with living systems or sophisticated AI," said co-author Yoshikatsu Hayashi, a biomedical engineer at the University of Reading in the UK. "This opens up exciting possibilities for developing new types of 'smart' materials that can learn and adapt to their environment."

Hydrogels are soft, flexible biphasic materials that swell but do not dissolve in water. So a hydrogel may contain a large amount of water but still maintain its shape, making it useful for a wide range of applications. Perhaps the best-known use is soft contact lenses, but various kinds of hydrogels are also used in breast implants, disposable diapers, EEG and ECG medical electrodes, glucose biosensors, encapsulating quantum dots, solar-powered water purification, cell cultures, tissue engineering scaffolds, water gel explosives, actuators for soft robotics, supersonic shock-absorbing materials, and sustained-release drug delivery systems, among other uses.

In April, Hayashi co-authored a paper showing that hydrogels can "learn" to beat in rhythm with an external pacemaker, something previously only achieved with living cells. They exploited the intrinsic ability of the hydrogels to convert chemical energy into mechanical oscillations, using the pacemaker to apply cyclic compressions. They found that when the oscillation of a gel sample matched the harmonic resonance of the pacemaker's beat, the system kept a “memory” of that resonant oscillation period and could retain that memory even when the pacemaker was turned off. Such hydrogels might one day be a useful substitute for heart research using animals, providing new ways to research conditions like cardiac arrhythmia.

For this latest work, Hayashi and co-authors were partly inspired by a 2022 study in which brain cells in a dish—dubbed DishBrain—were electrically stimulated in such a way as to create useful feedback loops, enabling them to "learn" to play Pong (albeit badly). As Ars Science Editor John Timmer reported at the time:

Pong proved to be an excellent choice for the experiments. The environment only involves a couple of variables: the location of the paddle and the location of the ball. The paddle can only move along a single line, so the motor portion of things only needs two inputs: move up or move down. And there's a clear reward for doing things well: you avoid an end state where the ball goes past the paddles and the game stops. It is a great setup for testing a simple neural network.

Put in Pong terms, the sensory portion of the network will take the positional inputs, determine an action (move the paddle up or down), and then generate an expectation for what the next state will be. If it's interpreting the world correctly, that state will be similar to its prediction, and thus the sensory input will be its own reward. If it gets things wrong, then there will be a large mismatch, and the network will revise its connections and try again.

There were a few caveats—even the best systems didn't play Pong all that well—but the approach mostly worked. Those systems comprising either mouse or human neurons saw the average length of Pong rallies increase over time, indicating they might be learning the game's rules. Systems based on non-neural cells, or those lacking a reward system, didn't see this sort of improvement. The findings provided some evidence that neural networks formed from actual neurons spontaneously develop the ability to learn. And that could explain some of the learning capabilities of actual brains, where smaller groups of neurons are organized into functional units.