ControlNet is a landmark open-source project with 34,000+ stars on GitHub that taught diffusion models to follow precise spatial guidance — edges, poses, depth and more.
Text prompts alone give you little control over composition. ControlNet, created by lllyasviel, changed that in 2023 by adding a trainable control branch to Stable Diffusion: you feed the model a conditioning image — a canny edge map, a depth map, a human pose skeleton, a segmentation mask — and generation follows that structure while the prompt controls style and content. The repository collects the original models and training code that ignited an entire ecosystem of controllable generation tools, and its techniques are now built into virtually every serious image-generation workflow, from research pipelines to production design tools. The codebase is a reference implementation rather than an actively evolving product — the repository was last updated in February 2024 — but it remains the canonical starting point for understanding controllable diffusion.
Its cultural footprint is hard to overstate: within months of release, ‘ControlNet’ became the generic name for guided generation, and the paper behind it collected awards while the repo collected tens of thousands of stars. Architects use it to hold a building’s massing constant while restyling facades, game artists lock character poses across a sprite sheet, and product designers keep silhouettes fixed while exploring surface treatments. Even as newer model families ship native control mechanisms, they are almost all measured against — and often trained in the image of — this repository.
ControlNet is free and open source under the Apache License 2.0. Running it requires your own GPU or a hosted diffusion environment.
Researchers, image-generation power users and tool builders who need precise, repeatable control over diffusion output rather than prompt-only generation.