llmfit

llmfit is an open-source command-line tool with 38,000+ stars on GitHub that answers a deceptively simple question: which open-source LLMs can your hardware actually run — and how well?

Overview

Model cards advertise parameter counts, but what decides your experience is whether a given quantization fits your VRAM and how fast it will generate. llmfit inspects your machine — CPU cores, system RAM, discrete and integrated GPUs, VRAM and unified-memory architecture across NVIDIA CUDA, Apple Silicon, AMD ROCm and Intel OneAPI — then runs its compatibility engine over hundreds of models and providers. For each candidate it projects the memory footprint and tokens-per-second across GGUF, AWQ, GPTQ and EXL2 quantizations, scoring quality, speed and fit. You drive it from a zero-dependency terminal UI, a web dashboard, or a REST API that deployment pipelines can query. Its newest trick is measured truth: download a model, serve it, benchmark real tokens-per-second on your machine, and contribute the result back with a pull request, so users on identical hardware see measured numbers instead of estimates. The project is under very active development as of October 2026.

Key Features

• Hardware auto-detection – CPU, RAM, GPUs and VRAM across CUDA, Apple Silicon, ROCm and OneAPI.
• Model compatibility engine – fit projections for hundreds of models across major quantization formats.
• Quality, speed and fit scores – ranked recommendations instead of raw spec tables.
• Benchmark and share – measure real tokens-per-second and contribute results back to the project.
• Three interfaces – interactive TUI, web dashboard and a REST API for automation.
• Cross-platform – macOS (Apple Silicon and Intel), Linux (x86_64 and ARM64) and Windows.
• One command – a single invocation scans your system and prints what fits.

💳 Pricing

llmfit is completely free and open source under the MIT license. It runs locally and needs no account, API key or telemetry.

Who Is It For

Local-LLM enthusiasts, homelab builders and developers choosing models for a specific machine — anyone tired of downloading a 40 GB model only to discover it cannot fit.