BitNet

BitNet is Microsoft’s official inference framework for 1-bit large language models, with 41,000+ stars on GitHub, built to run LLMs fast and losslessly on ordinary CPUs.

Overview

BitNet starts from a radical idea: train the model with ternary weights — each weight is only -1, 0 or +1 — so a “1.58-bit” model needs a fraction of the memory of a 16-bit model and can be evaluated with additions instead of expensive multiplications. This repository (bitnet.cpp) is the official inference engine for that model family, with hand-optimized kernels for x86 and ARM CPUs and, since 2025, official GPU kernels as well. Microsoft has released official models to run on it, including the BitNet b1.58 2B model trained on 4 trillion tokens, and in 2026 the first 1-bit embedding models, which the project reports run 1.4–2.3 times faster than F16 equivalents on CPU prefill. For edge and local AI, it points to a future where capable models run on laptops and small devices without a GPU. The project is actively maintained, with releases continuing through 2026.

Key Features

• bitnet.cpp inference engine – dedicated runtime for ternary 1-bit LLMs, optimized for CPU first.
• Tiny memory footprint – roughly 1.58 bits per weight instead of 16, shrinking models dramatically.
• Lossless low-bit inference – designed to preserve output quality at about 2 bits per weight.
• Official models included – run the BitNet b1.58 2B model and 1-bit embedding models from Hugging Face.
• GPU kernels – official GPU inference support extends 1-bit models beyond CPUs.
• Cross-platform kernels – tuned implementations for x86 and ARM processors.
• Edge-ready – targets laptops, embedded devices and other hardware without accelerators.

💳 Pricing

BitNet is completely free and open source under the MIT license, including the framework and Microsoft’s official BitNet models. Your only cost is the hardware you run it on.

Who Is It For

Researchers exploring ultra-low-bit model architectures, and developers who want to run capable LLMs on CPUs, laptops and edge devices where a GPU is not an option.