Transformer Debugger is an open-source interpretability tool from OpenAI’s Superalignment team, released on GitHub where it has earned about 4,122 stars. It was built to investigate exactly why a small language model produces one token instead of another. By combining automated interpretability techniques with sparse autoencoders, the tool lets researchers explore model internals interactively — often before writing any code — and trace the circuits responsible for a specific behavior.
Inside Transformer Debugger you can see which neurons, attention heads, and autoencoder latents contribute most to a chosen behavior on a given prompt. Components come with automatically generated explanations of what makes them activate, and you can intervene directly in the forward pass — ablating or steering a component — and immediately observe how the output changes. Connections between components can be followed across layers, which helps researchers discover the step-by-step circuits a model uses for tasks like indirect object identification in GPT-2 small. Linked video walkthroughs demonstrate complete investigations step by step, which makes the workflow considerably easier to learn.
Transformer Debugger is free and open source under the MIT license. Anyone can download it from GitHub and use it for research; the example workflows rely on publicly available model investigation datasets.
Transformer Debugger is for AI safety and interpretability researchers, and for advanced students who want to understand how transformer language models actually compute their outputs rather than treating them as black boxes.