KAI Scheduler is an open-source, Kubernetes-native scheduler built specifically for AI workloads at large scale. GPU clusters are expensive and usually shared between training jobs, inference services and interactive research, and the default Kubernetes scheduler treats GPUs as simple countable resources. KAI Scheduler adds the GPU-aware placement, fairness and queueing policies that large AI platforms need. The project has around 1,600 stars on GitHub and is under active development in 2026.
The scheduler understands GPU topology and workload shapes: it can pack jobs to reduce fragmentation, spread them for performance, guarantee fair shares between teams through hierarchical queues, and reclaim idle capacity from lower-priority work. It supports gang scheduling so distributed training jobs start all-or-nothing, time-based fairness for interactive sessions, and GPU sharing strategies that let smaller workloads co-locate. KAI installs alongside the standard Kubernetes scheduler and is operated through native Kubernetes resources, so platform teams manage AI capacity with familiar tooling.
KAI Scheduler is free and open source under the Apache 2.0 license.
Platform and MLOps teams running shared GPU clusters on Kubernetes who need fair, efficient scheduling for training and inference at scale.
KAI Scheduler was open-sourced by NVIDIA’s AI infrastructure teams, drawing on experience running some of the largest GPU clusters in the industry. It is designed to coexist with the default kube-scheduler rather than replace the whole control plane, and it ships with dashboards and metrics for operators. Workloads opt in through standard Kubernetes constructs, which makes it practical to trial on one namespace before rolling fair-share GPU scheduling out cluster-wide.