RL Baselines3 Zoo is a training framework built on top of Stable Baselines3 that packages reinforcement learning agents with tuned hyperparameters and ready-to-run training scripts. Maintained by the German Aerospace Center’s robotics group (DLR-RM), the GitHub project has around 2,900 stars and remains actively updated in 2026. It is the companion repository most SB3 users reach for when they want reproducible results instead of weeks of tuning.
The Zoo ships hyperparameter configurations that have already been optimized for a wide range of environments, including Atari, MuJoCo, PyBullet, and classic control tasks, together with a collection of pre-trained agents you can evaluate immediately. It integrates hyperparameter optimization with Optuna, experiment tracking, and utilities for recording videos of trained policies. Because everything is configuration-driven, you can train, tune, and benchmark agents with single commands.
The repository also serves as the project’s experiment archive: benchmark results are generated with fixed evaluation scripts so scores stay comparable across releases, and trained agents can be enjoyed interactively, rendering their behavior in the environment. It pairs with a companion hyperparameter-tuning workflow and clear documentation on the Read the Docs site. For teams standardizing on Stable Baselines3, the Zoo effectively defines the reference configurations everyone compares against.
RL Baselines3 Zoo is free and open source under the MIT license. There are no paid tiers; costs are limited to the compute you use for training.
Reinforcement learning researchers, robotics engineers, and students who use Stable Baselines3 and want proven configurations and baselines instead of starting every experiment from scratch.