Magnitude is an open-source inference engine that runs open-weight models as fast as your hardware allows. When you download a model, it automatically tunes its compute kernels on your exact chip — Apple Silicon, NVIDIA, AMD or plain CPU — and claims over 2x faster generation than Ollama on popular models. The project has about 7,200 stars on GitHub and is actively developed in 2026.
General-purpose local runners ship one-size-fits-all kernels; Magnitude instead spends up to a minute per model benchmarking different ways to split the math on your GPU and keeps the fastest combination that does not change the model’s output. Each model also ships with a tuned speculative-decoding drafter, a quantized KV cache that halves memory per agent session, and shared prefix caches so concurrent sessions do not slow each other down. Magnitude ships as a desktop app for macOS, Linux and Windows and is built for the agent era: one click connects local models to coding agents such as Claude Code, Codex, OpenCode and others, and anything else can use its OpenAI-compatible API. Everything runs on your machine — prompts, files and models never leave it, and no internet is needed once a model is downloaded. The engine is Apache-2.0 licensed and backed by Y Combinator.
Magnitude is free and open source under the Apache-2.0 license, with no token costs — you run models on your own hardware.
Magnitude is for developers and AI enthusiasts who run open models locally — especially as the engine behind coding agents — and want noticeably faster inference than general-purpose runners provide.