Versatile Diffusion is an open-source multi-flow diffusion model published on GitHub by SHI Labs that handles text-to-image, image-to-text and image variations in a single unified model. The project has gathered around 1,300 GitHub stars and was presented at ICCV 2023.
Most diffusion systems are built for one direction: text in, image out. Versatile Diffusion instead trains one network with multiple data flows, so the same model can generate an image from a caption, caption an image, and produce variations of an input image. It builds on latent diffusion with a multi-flow training scheme and a unified controller that switches behaviour per task, and the repository ships the official implementation, pretrained checkpoints and inference code accompanying the arXiv paper “Versatile Diffusion: Text, Images and Variations All in One Diffusion Model”. The code was last updated in 2023 and is best read as a research release: it shows how far a single diffusion backbone can be stretched across vision-language tasks, and it remains a reference for later unified generative models. A GPU is needed to run the checkpoints comfortably.
Versatile Diffusion is free and open source under the MIT license. You only need your own GPU compute to run it.
Generative-AI researchers and engineers studying unified vision-language diffusion models, and builders prototyping multi-task image generation.