oMLX is an open-source LLM inference server built specifically for Apple Silicon, managed right from the macOS menu bar. It has grown quickly to nearly 23,000 stars on GitHub, is under active development with updates in October 2026, and is available under the Apache-2.0 license.
Built on Apple’s MLX framework, oMLX turns a Mac into a capable local inference server. Its two headline techniques are continuous batching, which lets the server handle concurrent requests efficiently instead of one prompt at a time, and SSD caching, which offloads part of the memory burden to fast local storage so larger models fit on machines with limited RAM. A lightweight menu-bar app makes starting, stopping, and configuring the server feel like using a native Mac utility rather than a developer tool.
In practice, oMLX fits the way people already use local models: start the server once from the menu bar, then point your chat client, coding assistant, or scripts at its local API endpoint and work as if you were calling a cloud service, except everything stays on your Mac. Continuous batching means interactive chat and background batch jobs can share the same server gracefully, while the SSD cache keeps frequently used models ready without exhausting memory. For Apple Silicon owners, it removes most of the friction that usually comes with self-hosting open-weight models.
oMLX is free and open source under the Apache-2.0 license. There is no subscription or hosted plan; the only cost is your own Mac hardware.
oMLX is for Mac users and developers who want a fast, private, always-available local LLM server on Apple Silicon, whether for coding assistants, experimentation, or powering local AI workflows.