Intel Neural Compressor is an open-source Python library for model compression, published on GitHub with about 2,700 stars. It applies quantization, pruning and knowledge distillation to deep learning models so they run faster and cheaper, with particular strength in low-bit LLM quantization.
The library implements state-of-the-art techniques including static and dynamic quantization, SmoothQuant, GPTQ, AWQ and weight-only quantization down to INT4 and emerging FP4/MXFP4 formats, alongside sparsity and quantization-aware training. It works across PyTorch, TensorFlow, JAX and ONNX Runtime, and supports popular model families such as LLaMA, Qwen and DeepSeek as well as vision-language models. An auto-tuning engine searches for the best compression recipe under an accuracy budget. The project is Apache-2.0 licensed and actively maintained by Intel in 2026. Extensive documentation, example notebooks for popular models and a model zoo of pre-compressed checkpoints help users apply the techniques without starting from scratch. It also integrates with popular serving stacks so compressed models can be deployed directly into production inference pipelines.
Intel Neural Compressor is completely free and open source under the Apache-2.0 license. There are no paid tiers, hosted versions or usage limits.
Intel Neural Compressor is for ML engineers deploying models on CPU or edge hardware who need smaller, faster models without giving up much accuracy.