KServe is an open-source, CNCF-hosted platform for standardized distributed generative and predictive AI inference on Kubernetes. It has gathered about 6,100 stars on GitHub, is licensed under Apache-2.0, and is actively developed with updates in 2026.
KServe defines a standard InferenceService resource for Kubernetes, so deploying a model feels the same whether it is a scikit-learn classifier, a PyTorch network or a large language model served with vLLM. It builds on Kubernetes, Knative and Istio to provide serverless-style inference with autoscaling, including scale-to-zero, canary rollouts and request-based scaling for GPUs. For generative AI it supports LLM serving patterns such as KV-cache-aware routing, and for predictive ML it covers frameworks including TensorFlow, XGBoost and ONNX, plus explainability and outlier detection integrations.
KServe also standardizes the serving protocol itself through the Open Inference Protocol, so clients and model servers from different vendors can interoperate, and inference graphs let you chain models with ensemble and transformer steps. Storage initializers pull models from cloud object stores, Hugging Face and other sources at deploy time. Because it is a CNCF sandbox project with contributions from companies across the industry, KServe has become a common default choice for organizations standardizing ML serving on Kubernetes.
KServe is free and open source under the Apache-2.0 license. There is no paid edition; your costs are limited to the Kubernetes infrastructure you run it on.
KServe is for ML platform engineers and MLOps teams who need a consistent, scalable way to serve both classical models and large language models in production Kubernetes environments.