Researchers Unlock Biological AI Transformer Theory

KEY POINTS

• Researchers explore transformers’ biologically plausible construction.
• Astrocytes and neurons could perform a transformer’s core computation.
• New study may bridge gap between neuroscience and machine learning.

Artificial neural networks, models prevalent in machine learning, are termed so due to their inspiration from biological neurons in the human brain. A relatively newer model, the transformer, has exhibited impressive performance capabilities. Examples of such systems include ChatGPT and Bard. The inherent construction and operation of transformers have been ambiguous, especially in drawing parallels with biological counterparts. A collaboration involving researchers from MIT, the MIT-IBM Watson AI Lab, and Harvard Medical School postulates that transformers could be constructed biologically using a combination of neurons and astrocytes.

Studies indicate that astrocytes, cells abundantly found in the brain but are not neurons, have interactions with neurons. They contribute to various physiological functions, like blood flow regulation. Their computational role remains undefined. The research, published in Proceedings of the National Academy of Sciences, delves into the computational potential of astrocytes. The researchers have formulated a mathematical model highlighting the potential collaboration between neurons and astrocytes in creating a biologically viable transformer.

Neurons communicate using chemicals termed neurotransmitters. In certain communication instances, an astrocyte, which can form millions of tripartite synapses, gets involved. These astrocytes absorb some neurotransmitters and can, at intervals, signal back to the neurons. Their slower operational nature allows them to retain and integrate neuron-communicated information, effectively forming a memory buffer. This process paves the way for them to be integral components in transformer computations.

The researchers consolidated their insights into a mathematical model that emulates the workings of a transformer. They juxtaposed core transformer mathematics with biophysical neuron-astrocyte interaction models, leading to a defining equation. Numerical simulations confirmed the theoretical model’s effectiveness, setting the stage for subsequent experimental validation. One major implication hints at astrocytes possibly playing a role in long-term memory, given their information storage capability.

While the study offers a promising bridge between computational neuroscience and artificial intelligence, more exploration is necessary. Should these postulations hold, a significant surge in research dedicated to astrocytes in computational neuroscience is anticipated.