TRL (Transformer Reinforcement Learning) is Hugging Face’s open-source library for training transformer language models with reinforcement learning and preference optimization. It has more than 19,500 stars on GitHub, is actively maintained with updates in October 2026, and is released under the Apache-2.0 license.
TRL covers the full post-training stack that turns a raw pretrained model into a helpful assistant. It starts with supervised fine-tuning, then adds the alignment methods the field relies on: reward modeling, PPO, Direct Preference Optimization (DPO), and GRPO, among others. Everything integrates tightly with the Hugging Face ecosystem, including Transformers, PEFT for parameter-efficient tuning, and Accelerate for distributed training, so a recipe that runs on one GPU scales to a cluster with minimal changes. It is the library behind many published open-model fine-tunes.
TRL’s value is that it standardizes recipes that are otherwise easy to get subtly wrong. Preference training involves data formatting, reference models, and stability tricks that differ between papers; TRL encodes the community’s best-known configurations and exposes them through documented trainers and command-line scripts. It supports the dataset formats practitioners already use, pairs naturally with the Hugging Face Hub for sharing models and preference data, and is routinely used to produce the fine-tuned checkpoints that accompany open model releases. For many teams, a TRL script is the shortest path from a base model to a well-behaved assistant.
TRL is completely free and open source under the Apache-2.0 license. Your only costs are the compute and any datasets you use for training.
TRL is for ML engineers and researchers who fine-tune and align open language models, whether building a domain-specific assistant, reproducing RLHF research, or teaching a base model to follow instructions and preferences.