Limits of Confidence in Diffusion

Researchers analyze discrete diffusion methods that generate multiple tokens per step, revealing that only conditionally independent token groups match training distributions. They demonstrate inherent limitations in modeling dependent token groups with per-position distributions, validated on a synthetic task showing significant deviations from expected distributions.