llama-swap

llama-swap is an open-source proxy, written in Go, that provides reliable model swapping for local LLM servers. It sits in front of any OpenAI- or Anthropic-compatible inference server such as llama.cpp or vLLM and loads and unloads models on demand. It has about 6,000 stars on GitHub, is MIT-licensed, and is actively maintained with updates in 2026.

Overview

Local inference servers usually load one model at a time, so switching models means restarting processes by hand. llama-swap removes that friction: clients keep talking to a single endpoint, and the proxy starts, stops and swaps the backing model processes according to the model requested in each call. It manages process lifecycles, health checks and request routing, and it works with any server that speaks the OpenAI or Anthropic API style, not just llama.cpp. The result is a simple way to offer a whole shelf of local models through one stable address while only one (or a configured few) occupy memory at a time.

Key Features

• Single proxy endpoint that swaps local models on demand
• Works with llama.cpp, vLLM and any OpenAI/Anthropic-compatible server
• Automatic process management: start, stop, health-check and restart
• YAML configuration for models, groups and swap behavior
• Supports running multiple models simultaneously when resources allow
• Lightweight single binary written in Go, easy to self-host
• MIT-licensed and actively maintained

💳 Pricing

llama-swap is free and open source under the MIT license. There is no paid tier or hosted version; you run the binary on your own machine or server.

Who Is It For

llama-swap is for local-LLM enthusiasts and teams who run several open models on limited GPU memory and want clients to switch between them seamlessly without manual restarts.