TokenSpeed is an open-source LLM inference engine built for agentic workloads, published on GitHub with about 2,700 stars. Developed by Lightseek, it targets TensorRT-LLM-level performance with vLLM-level usability for production agent systems.
TokenSpeed is the first inference engine to separate the control plane from the execution plane: a C++ control plane encodes request lifecycles and KV cache ownership as a finite-state machine, using the type system to enforce safe resource reuse at compile time, while a Python execution plane runs the models. Its pluggable kernel system includes one of the fastest MLA implementations on NVIDIA Blackwell hardware, and the roadmap spans current accelerators from NVIDIA, AMD and Intel. The engine serves modern open models such as DeepSeek, Qwen, GLM and Kimi, and was welcomed into the PyTorch ecosystem landscape. It is MIT licensed and under active development in 2026. The repository documents its scheduler design, kernel registry and supported model list in detail, positioning the engine specifically for long, multi-turn agent sessions rather than single-shot chat serving.
TokenSpeed is completely free and open source under the MIT license. There are no paid tiers; you deploy the engine on your own GPU infrastructure.
TokenSpeed is for inference engineers and platform teams serving LLMs for agent products who need maximum tokens per second without giving up usability.