BentoML is an open-source platform for serving AI models and applications in production. It turns models into reliable inference APIs with the packaging, scaling and deployment machinery handled for you, and supports everything from classic machine-learning models to large language model apps and multi-model pipelines. The project has about 8,900 stars on GitHub and is actively developed in 2026.
With BentoML you define services in Python, and the framework builds optimized, containerized deployments (“Bentos”) that can run anywhere. It handles model inference APIs, asynchronous job queues for batch-style workloads, LLM serving with streaming, and pipelines that chain several models together. Features such as adaptive batching, autoscaling and GPU support help production workloads stay fast and cost-efficient, and the same code can be deployed on your own infrastructure or on BentoCloud, the project’s managed platform. BentoML works with models from all the major frameworks โ PyTorch, TensorFlow, scikit-learn, Hugging Face Transformers and Diffusers among them โ and includes built-in support for model management, request tracing and metrics. Its declarative configuration lets teams pin down dependencies and hardware requirements once, then reproduce the same service anywhere, which is why it shows up in production stacks ranging from startups to large platforms serving millions of inference requests.
BentoML itself is free and open source under the Apache-2.0 license for self-hosting. BentoCloud, the managed deployment platform from the same team, is a paid service priced by usage.
BentoML is for ML engineers and platform teams who need to ship models โ especially LLM applications โ as scalable, observable production services without building serving infrastructure from scratch.