Transformers Model Integration Contracts

On July 30, 2026, Transformers integrated PyTorch's scaled dot-product attention for T5, inadvertently causing UMT5's decoder to access future tokens during training. This bug persisted unnoticed in the main branch for nearly two months until detected by a small UMT5 model on CPU. To prevent such issues, Transformers proposes Model Integration Contracts (MIC), which use small fixture models and baseline tests to catch regressions early in development.