LLM Compressor is an open-source library from the vLLM project for compressing large language models so they run faster and cheaper in production. Published on GitHub, it has gathered about 3,895 stars and is under active development in 2026. Note that this is the vLLM project’s compression toolkit โ a distinct project from another product with a similar name that is also listed in this directory. Its job is to take a full-precision model and produce an optimized version ready for high-throughput inference with vLLM.
The library implements state-of-the-art compression research in a unified, production-oriented toolkit: weight and activation quantization with methods such as GPTQ, AWQ, and FP8, smooth quantization techniques, and sparsity and pruning for supported hardware. One-shot calibration workflows keep the process practical โ you provide a small dataset, run a compression recipe, and export a model in the formats vLLM serves natively. Recipes and examples cover popular open model families, and the project tracks current research so new algorithms land in the same consistent interface. The result is lower memory use and higher serving throughput without retraining the model from scratch.
LLM Compressor is free and open source under the Apache-2.0 license. There are no paid tiers; the savings it creates come from the smaller, faster models you can then serve on your own infrastructure.
LLM Compressor is for ML engineers and platform teams deploying open models at scale, where GPU memory and inference cost matter โ anyone who needs production-grade quantization and compression that plugs directly into a vLLM serving stack.