ToolBench is an open platform from OpenBMB for training, serving and evaluating large language models on tool use. Published alongside an ICLR 2024 spotlight paper, it provides the data, training pipelines and evaluation harness behind ToolLLM, one of the early open efforts to teach models to call real-world APIs reliably. It has collected more than 5,700 stars on GitHub.
ToolBench is built around more than 16,000 real-world APIs collected from RapidAPI. It includes instruction-tuning data generated for those APIs, a depth-first search tree reasoning strategy for multi-step tool calling, fine-tuning recipes that produced the ToolLlama model family, serving code, and an automatic evaluator that measures whether tool use actually succeeded. Together these pieces cover the complete loop of building a tool-using model, from raw API documentation to a benchmarked result. The dataset and evaluation protocol became a reference point for later tool-use research, and the repository remains a practical starting point for fine-tuning open models into competent API callers. Because everything from data generation prompts to the evaluator is included, results reported with ToolBench are considerably easier to reproduce than results from closed tool-use benchmarks. The project is maintained by OpenBMB, a large open-source AI community, and continues to attract citations and derivative work.
Free. ToolBench is an open research platform released under the Apache-2.0 license; you only need your own compute and API access to use it.
Researchers and engineers who train or benchmark language models on tool use, and who need realistic API-calling data and a reproducible evaluation harness.