Rapid-MLX is an open-source LLM inference server for Apple Silicon, built on Apple’s MLX framework and developed on GitHub, where it has gathered about 4,000 stars. It serves local models through OpenAI- and Anthropic-compatible APIs and is engineered to be substantially faster than Apple’s own mlx-lm server on the same hardware and weights.
Rapid-MLX focuses on the workload that matters most for local AI today: coding agents and apps issuing tool calls against a model running on a Mac. It adds continuous batching so one machine can answer several clients at once, speculative decoding for higher throughput, prompt caching for near-instant follow-up turns, and a large set of tool-call parsers with automatic recovery when quantized models garble a call. It can check whether a Hugging Face model fits your Mac before downloading, convert checkpoints with one command, and serve text, vision, audio and video models. The project is updated very actively, with releases and benchmarks published through late 2026.
Rapid-MLX is free and open source under the Apache 2.0 license, including both the engine and the desktop app. There are no accounts, subscriptions or cloud inference charges; models run entirely on your own Mac.
Rapid-MLX is for Mac users and developers who want a fast, private local LLM endpoint for coding agents like Claude Code or Cursor, and for teams turning a spare Mac into a shared on-premise inference server.