Pruning LLMs Like a Physicist: Block Removal as an Ising Optimization Problem

Researchers reformulated the problem of pruning transformer blocks in large language models as an Ising spin system optimization, enabling better selection of blocks to remove. Applying this to Llama-3.3-70B-Instruct at 50% compression improved MMLU benchmark scores by nearly 23 points over prior methods. This approach leverages physics-inspired solvers to handle the combinatorial complexity of block interactions during pruning.