SGLang-Omni is a high-performance serving runtime for interactive, multimodal model systems, from the team behind the widely used SGLang inference framework. Where classic LLM servers focus on text, SGLang-Omni is built for real-time systems that mix speech recognition, text-to-speech, audio and music generation and language modeling in one pipeline. It is an open-source project on GitHub with around 1,600 stars, under very active development in 2026.
Interactive AI products such as voice agents need several models working together with tight latency budgets: transcribing speech, reasoning, and synthesizing a spoken reply while the user is still engaged. SGLang-Omni provides a runtime for composing those stages into one serving system, with streaming throughout, an OpenAI-compatible API surface, and the throughput optimizations the SGLang project is known for, including efficient batching and CUDA-accelerated execution. Pipelines are defined in PyTorch and can be distributed across GPUs for production traffic.
SGLang-Omni is free and open source under the Apache 2.0 license. You provide your own GPU infrastructure.
Engineers building real-time voice agents and interactive multimodal products who need one runtime for the whole speech-to-speech pipeline.
SGLang-Omni extends the SGLang project’s RadixAttention and continuous batching ideas from text-only serving to full multimodal pipelines, where a spoken reply depends on a chain of models rather than one. By keeping every stage inside one runtime, it avoids the serialization and network hops that appear when separate ASR, LLM and TTS servers are glued together, which is where much of the latency in voice agents usually hides.