Sakana AI Researchers Introduce PC-ALM, a Layer-Local Alternative to Backpropagation That Trains 1000-Layer Networks

Sakana AI researchers developed PC-ALM, a layer-local training method that approximates backpropagation's performance on deep residual MLPs up to 1000 layers, tested on MNIST. PC-ALM uses augmented Lagrangian techniques to overcome predictive coding's limitations in deep, narrow networks, enabling local updates without global phase locking.