verl is a flexible, production-ready reinforcement learning training library for large language models, initiated by ByteDance’s Seed team. It is the open-source implementation of the HybridFlow framework, and it has become a go-to choice for RL post-training: algorithms such as GRPO and PPO can be expressed in a few lines of code, while the engine scales to models of hundreds of billions of parameters across hundreds of GPUs. The project has close to 24,000 stars on GitHub.
RL post-training is where modern reasoning models are made, but the infrastructure is notoriously hard: training, generation, reward computation and data movement must interleave efficiently. verl’s hybrid-controller programming model decouples computation from data dependencies, so researchers can describe complex RL dataflows flexibly while the system executes them efficiently. It integrates with the standard stack rather than replacing it: FSDP and Megatron-LM for training, vLLM, SGLang and Hugging Face Transformers for rollout generation, and tracking tools such as Weights & Biases and MLflow. It supports a long list of algorithms (PPO, GRPO, DAPO, ReMax, RLOO and more), vision-language and multimodal RL, LoRA-based RL, and hardware from NVIDIA, AMD and Ascend. The DAPO recipe, trained with verl, surpassed DeepSeek’s GRPO results on AIME 2024, and verl has powered models such as Doubao-1.5-pro.
verl is free and open source under the Apache-2.0 license. There is no paid tier; costs are limited to the compute you run it on.
verl is for research labs and engineering teams doing RL post-training of language and multimodal models, especially those who need research flexibility without giving up production-grade throughput and scalability.