nanoGPT is Andrej Karpathy’s celebrated open-source repository — nearly 64,000 stars on GitHub — billed as the simplest, fastest codebase for training and fine-tuning medium-sized GPTs.
nanoGPT distills a GPT implementation down to two readable files: a compact model definition and a training loop, deliberately stripped of framework bloat so students can understand every line. Despite the minimalism it is genuinely capable: it reproduces GPT-2-class training, supports fine-tuning pretrained checkpoints, and has served as the reference implementation for countless courses, experiments and speedrunning records for the fastest GPT-2 training. A character-level Shakespeare demo trains in minutes on a laptop, giving newcomers a complete, working language model almost immediately. The repository has trended 61 times in the archive and, while its last push was in November 2025, it remains the canonical teaching implementation of the transformer training stack, released under the MIT license.
The repository is also a piece of open-source history: Karpathy’s line-by-line video lectures built around this codebase taught hundreds of thousands of people how transformers actually work. Its descendant projects and speedrun community keep pushing the same minimal code faster, which makes nanoGPT both a teaching artifact and a living benchmark for training efficiency research.
nanoGPT is free and open source under the MIT license. You only need your own compute; small experiments run on a single consumer GPU or even a laptop.
Learners, educators and researchers who want to understand — and then modify — exactly how a GPT is trained, without fighting a large framework’s abstractions.