Nanotron is Hugging Face’s minimalist library for training large language models with 3D parallelism. The GitHub repository has collected around 2,840 stars and is actively maintained in 2026. Where many training frameworks hide the machinery behind layers of abstraction, Nanotron deliberately keeps the codebase small and readable, so engineers can see exactly how tensor, pipeline, and data parallelism fit together.
The library provides a compact implementation of distributed LLM training that combines data parallelism, tensor parallelism, and pipeline parallelism, with support for expert parallelism in mixture-of-experts models. It includes example training scripts, checkpointing, and configuration through simple dataclasses rather than sprawling config files. Nanotron has been used internally at Hugging Face for research on scaling and model architectures, and it pairs naturally with the wider Hugging Face ecosystem for tokenizers and datasets.
Nanotron is deliberately opinionated about simplicity: it avoids heavy frameworks, keeps parallel-state handling explicit, and ships unit tests that verify parallelism correctness on small models before you scale up. Example configurations cover dense decoder models in the style of Llama, and the codebase documents how sequence length, micro-batch size, and parallel degrees interact. That transparency has made it a favorite teaching codebase for engineers moving from single-GPU training to multi-node clusters.
Nanotron is free and open source under the Apache-2.0 license. You provide the GPU cluster; there is no commercial edition or hosted version.
ML engineers and researchers who train LLMs at scale and want a transparent, hackable training framework, as well as anyone learning how distributed training actually works under the hood.