AutoPrompt is an open-source framework for automatically tuning and calibrating prompts for large language models, built around a technique its authors call Intent-based Prompt Calibration. Instead of hand-crafting prompts through trial and error, you describe the intent of your task and provide a small calibration set, and the framework iteratively refines the prompt so the model’s outputs line up with what you actually meant. The project lives on GitHub, where it has gathered roughly 3,000 stars, and it remains actively maintained, with the latest updates in 2025.
Prompt quality is one of the biggest variables in any LLM application, yet prompt engineering is usually an unprincipled, manual process. AutoPrompt treats prompt tuning as an optimization problem: it analyzes where a model’s answers drift from the intent expressed in your examples, generates candidate prompt revisions, and keeps the versions that measurably improve accuracy on the calibration data. Because the calibration works at the intent level rather than on exact strings, the tuned prompts tend to generalize better than prompts overfit to a handful of examples. The framework is model-agnostic and can be pointed at popular hosted LLM APIs.
AutoPrompt is completely free and open source under the Apache-2.0 license. You only pay for whatever LLM API calls your own calibration runs make to your chosen model provider.
AutoPrompt suits ML engineers, prompt engineers, and researchers who maintain prompts in production and want a repeatable, measurable way to improve them instead of relying on intuition and spot checks.