Kimi K3 in C

Kimi K3 in C is an open-source inference engine that runs the 2.78-trillion-parameter Kimi K3 model on a single CPU in 8.24 GB of RAM. Written in portable C99 with no BLAS, no framework, and no GPU, it has earned more than 10,000 GitHub stars for showing that a frontier-scale mixture-of-experts model can be practical on consumer hardware.

Overview

Most large language models assume a data-centre GPU cluster, but Kimi K3 in C takes the opposite route. It is a dependency-free C program built from scratch that reads the Kimi K3 checkpoint, packs its shared trunk, and streams experts in and out of memory as needed. The project documents each reduction, from parsing checkpoint headers to reproducing the official tokenizer byte for byte. Setup is: clone, build, verify the machine, fetch the checkpoint, pack the trunk, and run. Documentation covers machines from 8 GB to 224 GB, with the RAM-versus-speed trade-off made explicit.

Key Features

• CPU-only inference: Run a 2.78-trillion-parameter model without a GPU, using a single CPU and as little as 8.24 GB of RAM.
• Zero dependencies: Portable C99 code with no BLAS and no machine-learning framework, keeping the build small and auditable.
• Mixture-of-experts routing: The engine selects 16 active experts out of 896 for each step, keeping computation focused while the full model stays available.
• Compact attention: Attention whose memory footprint does not grow with context, plus a latent design replacing ninety-six heads with one latent representation.
• Trunk packing and streaming: Ninety-three shared layers are packed for one read each, and an LRU cache manages experts so storage and memory can be traded against speed.
• A complete command-line tool: Prompt, memory, and generation options, a text chat mode, diagnostics, worked examples, benchmarks, and tests.
• Open and documented: Released under the Apache 2.0 licence, with long-form documentation on design, validation, and measurements.

💳 Pricing

Kimi K3 in C is free and open source under the Apache 2.0 licence. There is no paid tier in the repository; users provide their own machine and follow the documented steps to fetch and run the model checkpoint.

Who Is It For

Kimi K3 in C is for developers, systems programmers, and researchers who want to run a very large open model locally, study a from-scratch transformer implementation, or experiment with CPU inference. It also suits curious builders who would rather understand every layer than treat the model as a black box.