minGPT

minGPT is Andrej Karpathy’s minimal, readable re-implementation of the GPT language model in PyTorch: a complete, trainable generative pre-trained transformer in roughly 300 lines of code. It strips away framework complexity so that every part of the architecture, from token embeddings to the training loop, can be read and understood in one sitting. With about 25,000 stars on GitHub, it has become one of the most widely used teaching implementations of the transformer architecture.

Overview

Large language model frameworks are powerful but hard to learn from. minGPT takes the opposite approach: a small, clean, dependency-light codebase that still implements the real thing, including the transformer blocks, causal self-attention, and a training harness that can reproduce GPT-style results on character-level and small text tasks. It is deliberately educational, favoring clarity over features, and it has been used in university courses and countless self-study projects as the reference for how GPT actually works under the hood. Note that the README describes the project as semi-retired since January 2023, with Karpathy’s later nanoGPT repository recommended as its actively developed successor; minGPT remains valuable precisely because of its frozen simplicity.

Key Features

• Complete GPT implementation in about 300 lines of clean, readable PyTorch
• Covers embeddings, causal self-attention, transformer blocks and the full training loop
• Minimal dependencies and a small codebase that can be read end to end
• Example training scripts for character-level language modeling
• Widely used in courses and tutorials as the reference teaching implementation of GPT
• Pairs naturally with the successor nanoGPT project for a more complete training stack

💳 Pricing

minGPT is completely free and open source under the MIT license. There are no paid tiers or hosted services; you simply clone the repository and run it.

Who Is It For

minGPT is for students, educators and engineers who want to truly understand how GPT models work by reading and modifying real code, rather than treating the architecture as a black box inside a large framework.