NVIDIA Model Optimizer is a unified library of state-of-the-art model optimisation techniques โ quantisation, distillation, pruning, sparsity, neural architecture search and speculative decoding โ that compress deep learning models so they run faster and cheaper at inference time. It prepares models for deployment frameworks such as TensorRT-LLM, TensorRT and vLLM. The Apache-2.0 project, formerly known as TensorRT Model Optimizer, has more than 5,300 stars on GitHub and is under very active development in 2026.
Large models are expensive to serve, and the standard cure is to shrink them without giving back accuracy. Model Optimizer gives PyTorch and ONNX workflows a single toolkit for that job: post-training quantisation to formats such as FP8, INT8 and INT4 weight-only, sparsity and pruning to remove redundant weights, knowledge distillation to train smaller student models, and support for speculative decoding to accelerate generation. Optimised models export in formats that deployment engines understand, including checkpoints ready for TensorRT-LLM and vLLM, so the path from a Hugging Face model to a production endpoint stays short. Because it comes from NVIDIA, the library tracks the newest GPU architectures and the low-precision formats they accelerate, and it is used across NVIDIA’s own inference stack. For teams serving LLMs or vision models at scale, it is one of the standard tools for cutting latency and GPU cost per request.
Free. NVIDIA Model Optimizer is open source under the Apache-2.0 license.
ML engineers and inference teams who need to reduce the latency, memory footprint and serving cost of LLMs and vision models on NVIDIA hardware.