Tianshou is an elegant deep reinforcement learning library built on PyTorch, developed by researchers at Tsinghua University. It packages more than twenty modern RL algorithms into clean, readable, production-quality code, and has earned about 11,000 stars on GitHub with development continuing into 2026.
The library covers the full RL workflow: vectorized environment wrappers for fast parallel data collection, modular policy implementations, replay buffers, trainers for online, offline and imitation learning, and extensive logging. Algorithms include DQN variants, PPO, SAC, TD3, DDPG, TRPO, NPG, plus offline methods such as CQL and BCQ, all sharing a consistent API that makes swapping algorithms a one-line change. Tianshou is known in the research community for code clarity โ the implementations are heavily tested and documented, so papers’ results can be reproduced and then customized. It integrates with MuJoCo, Atari and custom Gymnasium environments out of the box. Beyond the algorithm collection, Tianshou’s collectors and replay buffers are engineered for throughput, gathering experience from many simulators in parallel through vectorized environments. The project maintains extensive documentation and tutorials that build from a first DQN to distributed training, plus a test suite that keeps published benchmark scores reproducible across releases. Development is led by Tsinghua’s machine learning group with a broad contributor base, and it has become a standard reference implementation cited across reinforcement learning courses and papers.
Tianshou is completely free and open source under the MIT license, for both research and commercial projects.
Tianshou is for RL researchers, students and engineers who want trustworthy PyTorch implementations they can read, reproduce and extend โ from coursework to robotics and game AI experiments.