The Design Space of Tri-Modal Masked Diffusion Models

Researchers from Cambridge, MIT, Apple, and DeepMind developed the first tri-modal Masked Diffusion Models pretrained from scratch on text, image-text, and audio-text data, analyzing scaling laws and optimizing training. Their 3B parameter model demonstrates strong performance in text generation, text-to-image, and text-to-speech tasks, advancing multimodal discrete diffusion modeling.