Sakana AI, the Tokyo-based artificial intelligence lab known for its biologically inspired approaches to machine learning, has published a paper extending backpropagation-free training to convolutional neural networks (CNNs) and reinforcement learning (RL) for the first time.
The paper, "Diffusing Blame," posted to arXiv on July 17, builds on the lab's earlier work on Error Diffusion — a learning rule that updates network weights without the backward pass of gradients that underpins virtually every modern deep learning system.
Why It Matters
Backpropagation — the algorithm that computes gradients by propagating error backward through a network — is computationally expensive and is considered biologically implausible by many neuroscientists. Real neurons do not send error signals backward along their axons. Error Diffusion offers an alternative: it spreads blame for mistakes forward through the network using a simple, local rule that more closely resembles how biological brains might learn.
Sakana's new work extends this approach to CNN architectures and to reinforcement learning settings for the first time, achieving 96.7% accuracy on MNIST and 61.7% on CIFAR-10 without any backpropagation. Their ED-PPO variant matches standard DFA-PPO on Brax reinforcement learning benchmarks and outperforms it on the more complex Craftax environment.
How It Works
The method keeps all learnable weights non-negative — a constraint known as Dale's principle in neuroscience, where real neurons either excite or inhibit but never both. It uses a dual-stream architecture with random feedback matrices, routing errors through a modulo scheme that keeps computation local and biologically plausible.
The 'Biology Tax'
The approach has a cost: networks trained with Error Diffusion use roughly 4x more parameters than their single-stream backpropagation equivalents. Sakana researchers openly call this the "biology tax" — the overhead of respecting biological constraints while maintaining competitive performance.
While Error Diffusion is not yet competitive with backpropagation on large-scale tasks like training GPT-scale language models, the extension to CNNs and RL represents a significant step toward learning algorithms that could one day run on neuromorphic hardware — chips designed to mimic the brain's architecture — at a fraction of the energy cost of today's GPUs.
