PrivateGPT

PrivateGPT is an open-source API layer for private AI applications — more than 57,000 stars on GitHub — that gives you RAG, tools and document intelligence on local models without sending data to the cloud.

Overview

Running a model locally is only step one; useful applications need retrieval, document parsing, tools and conversation state. PrivateGPT provides that complete backend as an open-source API, modeled on the Claude API style, that works with any OpenAI-compatible inference server and fully on-premise stacks. It offers production-grade RAG over your documents, skills and tool use, MCP integration, text-to-SQL and more, all designed so sensitive data never leaves your infrastructure. The project powers Zylon, the commercial on-premise AI platform used by enterprises, which keeps the open-source core battle-tested. It has trended 50 times in the archive, is Apache-2.0 licensed, and was actively developed in October 2026.

PrivateGPT was one of the first projects to prove, in mid-2023, that chatting with your documents on a fully local stack was practical; it has since matured from a demo into an API platform. Its architecture separates ingestion, retrieval and generation cleanly, which lets engineering teams swap models and vector stores as the local ecosystem improves without rebuilding their application.

Key Features

• Private RAG – ingest documents and chat over them entirely on local models.
• Complete API layer – embeddings, retrieval, generation, tools and skills behind one API.
• Inference-agnostic – works with any OpenAI-compatible server (llama.cpp, vLLM, Ollama and more).
• MCP and text-to-SQL – connect tools and query databases in natural language.
• Production heritage – the open core of Zylon’s enterprise on-premise platform.
• Document intelligence – strong parsing for PDFs and office files in the ingestion pipeline.

💳 Pricing

PrivateGPT is free and open source under the Apache License 2.0. The separate commercial Zylon platform, built for enterprise deployments, is sold by Zylon with its own pricing.

Who Is It For

Teams in regulated or privacy-sensitive environments — legal, healthcare, finance, government — who need real AI applications over their own documents with zero data leaving the building.