DLRover is an open-source system that automates distributed deep learning training, making large-model training jobs easier, more stable and faster to run. With about 1,700 stars on GitHub and active development in 2026, it handles the engineering side of cluster training so model developers can focus on model architecture instead.
Born inside Ant Group’s AI infrastructure work, DLRover provides automated operation and maintenance for deep learning jobs running on Kubernetes. It automatically scales training resources up and down with demand, detects failures and recovers from them without losing progress, and optimizes how cluster hardware is used โ which its authors describe as making training “green” as well as fast. It supports large language model training workloads and integrates with common deep learning frameworks, handling hardware acceleration and distributed execution details that otherwise consume platform engineering time. The goal is elastic, self-healing training that keeps expensive GPU clusters productively busy instead of idle or stuck on failed jobs.
DLRover grew out of large-scale production training inside Ant Group, so its design reflects the failure modes, scheduling problems and cost pressures of real industrial GPU clusters rather than small lab benchmarks.
DLRover is free and open source; there is no paid edition or hosted service from the project. Your costs are the cluster infrastructure you run it on, whether that is an on-premise Kubernetes deployment or cloud capacity you already pay for.
ML platform teams training large models on shared clusters, and research groups that lose time and GPU budget to failed or inefficient distributed training jobs. It is infrastructure for engineers, not a tool that end users interact with directly.