xFormers is a library of optimized, composable Transformer building blocks from Meta’s FAIR research team. Rather than shipping one fixed model, it provides fast, memory-efficient implementations of the pieces Transformers are made of โ attention mechanisms, fused kernels and sparse operations โ that researchers assemble into their own architectures. It has about 10,600 stars on GitHub and is actively maintained in 2026.
The library is best known for memory-efficient attention, which computes exact attention without materializing the full attention matrix, dramatically cutting GPU memory for long sequences; it has shipped production-grade integrations of Flash Attention-style kernels and many fused operations for normalization, activations and feed-forward blocks. Each component is hackable and benchmarked, and blocks can be mixed freely: pick an attention variant, a positional embedding and an MLP style, and xFormers wires them together with a factory API. These components power research models across Meta and the wider community, and the library is a common dependency when training or fine-tuning large vision and language Transformers on limited hardware. The library grew out of FAIR’s research programme on efficient Transformers and continues to track hardware trends, adding kernels for new GPU generations as they appear. Its components are individually importable, so a project can adopt just the attention kernel without committing to the whole system. Many popular training frameworks expose xFormers attention as a configuration flag, which makes its optimizations reachable even for teams that never import it directly.
xFormers is free and open source from Meta AI Research. There are no fees, tiers or hosted versions โ you install it from PyPI or source.
xFormers is for researchers and ML engineers training Transformers who need lower memory use and faster kernels, and who want to experiment with custom architectures from proven parts.