Olive

Olive is Microsoft’s open-source toolchain for preparing machine learning models for efficient on-device and cloud inference. Released under the MIT license with more than 2,400 GitHub stars and active development continuing into 2026, it automates the tedious chain of finetuning, conversion, quantization and optimization that stands between a trained model and a fast deployment.

Overview

Getting a large model to run well on a specific CPU, GPU or NPU usually means hand-tuning compression settings and conversion steps for each target. Olive replaces that trial and error with composable passes: you describe the model, the techniques to try — quantization, pruning, distillation, ONNX conversion — and the hardware target, and Olive searches combinations against your accuracy and latency budgets. It integrates with PyTorch, Hugging Face and ONNX Runtime, and supports techniques such as GPTQ, AWQ and LoRA finetuning as part of the same pipeline.

Key Features

• Automated model optimization across CPUs, GPUs and NPUs
• Composable passes for quantization, pruning, distillation and conversion
• Built-in support for popular methods such as GPTQ, AWQ and LoRA
• Searches technique combinations against accuracy and performance targets
• Works with PyTorch, Hugging Face Transformers and ONNX Runtime
• Produces ready-to-deploy ONNX models for edge and cloud runtimes

💳 Pricing

Olive is completely free and open source under the MIT license, maintained by Microsoft with no paid tier.

Who Is It For

Olive is aimed at ML engineers shipping models to production — especially on Windows devices, edge hardware and NPUs — who need smaller, faster models without sacrificing the accuracy their application depends on.