minbpe is Andrej Karpathy’s minimal, clean implementation of the Byte Pair Encoding (BPE) algorithm that powers the tokenizers inside GPT models. True to its name, it strips tokenization down to its essentials: a readable Basic implementation and a Regex-based implementation that mirrors GPT-4’s approach, together small enough to read in one sitting.
Tokenization is one of the least understood but most consequential parts of an LLM pipeline, and minbpe exists to make it click. The BasicTrainer builds merges by counting byte pairs directly, while the RegexBPETrainer adds the pre-tokenization splitting pattern used by production GPT tokenizers, plus handling of special tokens. Despite its teaching focus, the code is fully functional: you can train a tokenizer on your own corpus, encode and decode text, and save and load vocabularies. It accompanies Karpathy’s popular “Let’s build the GPT Tokenizer” lecture and pairs naturally with his nanoGPT and llm.c projects. The repository has more than 10,800 GitHub stars and is MIT-licensed; it was last updated in July 2024 and serves as a stable reference implementation.
minbpe is completely free and open source under the MIT license.
Students, engineers, and LLM builders who want to truly understand tokenization, or need a transparent BPE trainer for custom experiments.