PromptTools, from Hegel AI, is an open-source toolkit for testing and experimenting with prompts, models and vector databases in a rigorous, repeatable way. Released under the Apache-2.0 license, it has about 3,060 stars on GitHub and remains in development in 2026. It treats prompt development like software testing: define experiments, run them across models and configurations, and compare the results systematically instead of eyeballing outputs in a chat window.
The toolkit has two main parts. An experiment framework lets you sweep prompts, model parameters and providers (OpenAI, Anthropic, LLaMA-family local models and more), score outputs with automatic metrics or human feedback, and persist results for comparison. A Prompt Playground UI provides an interactive interface for trying prompts side by side. PromptTools also experiments against vector databases such as Chroma, Weaviate and LanceDB, so retrieval settings for RAG systems can be evaluated with the same discipline as the prompts themselves.
PromptTools was built by the team at Hegel AI out of frustration with manual prompt tweaking, and its design borrows from software engineering practice: experiments are code, runs are reproducible, and results can be diffed when a prompt changes. The playground runs locally, so prompts and data stay on your machine apart from the model calls themselves. For RAG builders, the vector-database experiments are particularly valuable, because chunking and retrieval choices often matter as much as the prompt, and PromptTools lets both be varied and scored inside the same experimental harness.
PromptTools is free and open source under the Apache-2.0 license. You pay only for any model API calls your experiments make.
LLM application developers and prompt engineers who want proper testing infrastructure for prompts and retrieval pipelines rather than ad-hoc manual checking.