GPUStack is an open-source GPU cluster manager for high-performance AI model serving. It orchestrates inference engines such as vLLM and SGLang across fleets of GPUs — on-premises, in Kubernetes, or in the cloud — and can also provision on-demand, SSH-accessible GPU instances. The project has about 5,900 stars on GitHub, is Apache 2.0 licensed, and was actively developed in 2026.
Serving LLMs at scale means juggling heterogeneous hardware, multiple inference engines and models that change weekly. GPUStack sits above that complexity: it discovers and manages GPU resources across clusters, automatically configures pluggable inference backends (vLLM, SGLang, TensorRT-LLM, or a custom engine), and exposes served models through OpenAI-compatible APIs. It supports GPUs from several vendors — NVIDIA CUDA, AMD ROCm and Ascend among them — and emphasizes day-0 support for newly released models, so freshly published weights can be deployed through the same control plane.
GPUStack is used for private and sovereign AI deployments where models cannot leave an organization’s own data centers. Its engine-pluggable design matters operationally: when a faster serving engine or a new model architecture appears, administrators can adopt it behind the same management layer instead of rebuilding their platform. Combined with multi-vendor GPU support — valuable when NVIDIA cards are scarce, or when already-owned AMD and Ascend hardware must be put to work — it targets the unglamorous but decisive layer of the AI stack: keeping fleets of accelerators busy, observable and easy to serve from.
Free. GPUStack is open source under the Apache 2.0 license; you run it on your own GPU infrastructure.
Platform and infrastructure teams running private LLM inference who need one control plane for GPU fleets, engines and model deployments.