FreeToken

FreeToken is an open-source, edge-native inference engine with 15,097 stars on GitHub. It brings datacenter-scale model serving to your desktop, running 290B+ frontier mixture-of-experts models locally on consumer hardware at interactive speeds.

Overview

Frontier open-weight models have outgrown the hardware most people own, and cloud inference means sending your prompts and data to someone else. FreeToken closes that gap by treating the heterogeneous resources of a personal machine, including GPUs, CPUs, host memory, and interconnects, as one unified, elastic inference platform. Its edge-native mixture-of-experts runtime uses bandwidth-adaptive CPU-GPU co-execution, full-layer double-buffered prefill streaming, global expert caching, and the FTW fast weight format to serve massive models on gaming PCs and workstations. It supports frontier open-weight MoE models such as DeepSeek-V4-Flash, Qwen3.6-35B-A3B, and GLM-5.2 across quantization formats including MXFP4, NVFP4, FP8, and BF16, and exposes Anthropic and OpenAI-compatible APIs so it plugs straight into coding and tool-calling agents such as Codex, Claude Code, OpenCode, and OpenClaw. A desktop app for Windows and Linux sets the engine up for you with a GUI for running models, chatting, and tuning, while a CLI covers scripted and server-style use.

Key Features

• Run 290B+ frontier MoE models locally on gaming desktops, workstations, and laptops.
• Edge-native runtime with bandwidth-adaptive CPU-GPU co-execution and double-buffered prefill streaming.
• Semantic-aware caching with anchor checkpoints and KV caches that avoid recomputation when agentic context is edited.
• Elastic memory management that reallocates VRAM between expert caches and KV memory without restarts.
• Support for DeepSeek, Qwen, and GLM MoE models across MXFP4, NVFP4, FP8, and BF16 quantization.
• Anthropic and OpenAI-compatible APIs for seamless integration with coding agents and agent harnesses.
• Native support for NVIDIA RTX 30, 40, and 50 series GPUs, plus a desktop GUI and CLI.

💳 Pricing

FreeToken is completely free and open source under the Apache 2.0 license. The desktop app and CLI cost nothing, and because models run on your own hardware, there are no per-token fees beyond the electricity your machine uses.

Who Is It For

FreeToken is built for developers, researchers, and AI enthusiasts who want to run large open models privately and cheaply. It suits anyone with a strong gaming GPU who is tired of API bills, agent builders who need a local model endpoint, and privacy-minded users who want frontier-class models without sending prompts to the cloud.