Bitsandbytes

Bitsandbytes is the open-source quantization library that made large language models practical on consumer GPUs. Published on GitHub with around 8,500 stars and maintained by the Bitsandbytes Foundation, it provides the k-bit quantization and 8-bit optimizer technology that much of the open LLM ecosystem is built on.

Overview

The library is best known for two breakthroughs. LLM.int8() introduced mixed-precision decomposition that lets models be loaded and run in 8-bit with almost no quality loss, halving memory requirements. Its 4-bit NormalFloat (NF4) quantization is the foundation of QLoRA, the technique that lets a single gaming GPU fine-tune models with tens of billions of parameters. Bitsandbytes also includes 8-bit versions of popular optimizers such as AdamW, which cut the memory used by training states. Through its tight integration with Hugging Face Transformers and PEFT, loading a model in 8-bit or 4-bit is often a one-line change, which is why bitsandbytes ships inside countless fine-tuning notebooks, local LLM apps, and research codebases. Development is very active, with repository updates continuing through late 2026.

Key Features

• 8-bit LLM.int8() inference with minimal accuracy degradation
• 4-bit NF4 quantization powering the QLoRA fine-tuning method
• Memory-efficient 8-bit optimizers including AdamW and SGD
• One-line integration with Hugging Face Transformers and PEFT
• Supports a wide range of NVIDIA GPUs for local training and inference
• Actively maintained foundation project used across the LLM ecosystem

💳 Pricing

Bitsandbytes is free and open source under the MIT license. It is a library you install from PyPI or GitHub; there is no paid tier, cloud service, or usage fee.

Who Is It For

Bitsandbytes is essential for anyone fine-tuning or running LLMs on limited hardware — hobbyists with a single GPU, researchers reproducing QLoRA results, and engineers shrinking models to fit production memory budgets.