GLIGEN is the official open-source implementation of the CVPR 2023 paper on open-set grounded text-to-image generation, with 2,200+ stars on GitHub — it lets diffusion models place objects exactly where you tell them to.
A text prompt alone gives you little control over layout: ask for a cat on the left and a dog on the right and a standard text-to-image model will improvise. GLIGEN (Grounded-Language-to-Image Generation) adds grounding inputs to frozen text-to-image models — bounding boxes, keypoints and reference images — so generated scenes respect the composition you specify while keeping the model’s open-set generality: it can ground concepts it was never explicitly trained to place. The paper reports zero-shot performance on COCO and LVIS that beats existing supervised layout-to-image baselines by a large margin. The repository includes the model code and checkpoints, a project page and a Hugging Face demo, and GLIGEN was later integrated into LLaVA-Interactive for combined image chat, segmentation, generation and editing. Last updated in 2024, it remains a key reference for controllable image generation research.
GLIGEN is free and open source under the MIT license. You download the code and checkpoints and run them on your own GPU; there is no commercial service or paid tier attached.
Computer vision researchers and generative imaging developers who need layout control in text-to-image generation and want the reference implementation behind a widely cited grounding method.