X-Transformers is a PyTorch library by Phil Wang (lucidrains) that packages a complete, full-attention Transformer together with a large collection of experimental ideas from recent papers. It aims to be the one Transformer implementation a researcher needs: concise enough to read, complete enough to build real models with. The project has about 6,000 stars on GitHub, is MIT licensed, and was actively updated in 2026.
Rather than a monolithic model, X-Transformers provides composable building blocks — attention variants, memory mechanisms, positional schemes and more — behind simple Encoder, Decoder and TransformerWrapper classes. Many of lucidrains’ well-known paper reimplementations build on it, and it has become a standard prototyping substrate in the open-source research community: new attention ideas typically appear here quickly, clearly implemented and ready to mix into your own architecture experiments.
The library’s API keeps experiments terse: wrap a Decoder or Encoder in the TransformerWrapper class, declare dimensions and depth, and swap individual ideas in or out through constructor flags rather than forking the code. That configurability is why it shows up inside so many other lucidrains repositories and third-party reproductions — a researcher can test a new attention mechanism in an afternoon instead of a week. It also functions as a living reading list: skimming the features merged over the years is a quick tour of what the field has tried since the original Transformer paper, from rotary embeddings to various memory-augmented and sparse-attention schemes.
Free. X-Transformers is open source under the MIT license.
ML researchers and engineers prototyping Transformer variants, reproducing papers, or learning how modern attention architectures are put together.