LMCache is an open-source KV cache layer that makes large language model serving dramatically faster and cheaper by storing and reusing the key-value caches that inference engines normally recompute for every request. Born from University of Chicago research, it has earned about 12,100 stars on GitHub and is under very active development in 2026.
In LLM inference, much of the compute goes into re-encoding text the system has seen before โ long prompts, shared document contexts, earlier turns of a chat. LMCache intercepts that redundancy: it extracts KV caches from engines such as vLLM, SGLang and TGI, stores them across a tiered hierarchy of GPU memory, CPU memory and disk (or remote backends), and injects them back when the same content reappears, skipping the prefill work entirely. Its research lineage shows in the techniques involved: CacheGen compresses and streams KV caches efficiently, and CacheBlend fuses cached chunks for retrieval-augmented generation without full recomputation. The project positions itself as a ‘knowledge delivery network’ for LLM applications and reports speedups of up to roughly 8x at correspondingly lower cost in long-context and multi-turn scenarios. For serving teams, it is a drop-in layer: enable it alongside your engine and repeated-context workloads get faster without changing your models.
LMCache is free and open source under the Apache-2.0 license. You deploy it on your own serving infrastructure; the project’s website lists no paid tier, and your costs are the GPUs you already run.
LMCache is for inference and platform engineers serving LLMs at scale โ chatbots with long histories, document-processing pipelines, RAG services โ who want lower latency and GPU bills by caching computation instead of repeating it.