Bringing Nunchaku 4-bit Diffusion Inference to Diffusers

Nunchaku Lite, a 4-bit quantization method using SVDQuant, is now integrated into Diffusers, enabling faster and more memory-efficient diffusion model inference without extra compilation. Users can load pre-quantized models easily and quantize new architectures with the diffuse-compressor toolkit.