TensorFold is an open-source LLM inference engine that serves language models from a single Zig binary on Apple Silicon and NVIDIA GPUs — no Python, no MLX installation, no framework stack to babysit. It has about 1,430 stars on GitHub and is under active development in 2026.
Most local LLM serving stacks are layers of Python tooling. TensorFold takes the opposite route: one native binary that reads model checkpoints, tokenizes requests, and runs Metal kernels on Macs or CUDA kernels on NVIDIA hardware directly. You pull a checkpoint — the quick start uses a small 4-bit Qwen model — and serve it under a local name with an OpenAI-compatible API. On top of chat completions, the server also speaks completions, the Responses API, and Anthropic Messages, and it can serve a local checkpoint directory or an existing Hugging Face cache. There is also a TensorFold desktop workspace for macOS that bundles the engine with model downloads, hardware recommendations, live generation metrics, and document editing, for users who prefer an app over the command line. Model weights keep their own licenses; the engine itself is Apache-2.0.
TensorFold is free and open source under the Apache-2.0 license, including the engine and the desktop workspace beta. There is no paid tier; you run it on your own hardware with models you choose.
Developers and local-AI enthusiasts who want fast, self-contained LLM serving on a Mac or an NVIDIA machine — especially anyone tired of maintaining Python environments just to run inference, or building products on OpenAI-compatible local endpoints.