Speech-to-Speech is Hugging Face’s open-source pipeline for building realtime voice agents entirely from open models. It chains voice activity detection, speech recognition, a language model and speech synthesis into one streaming system you can talk to naturally, and it has gathered about 13,500 stars on GitHub with active development through 2026.
Commercial voice assistants hide their pipeline behind an API. This project shows how to assemble the same experience yourself from interchangeable parts: a VAD module notices when you start and stop speaking, a speech-to-text model such as a Whisper variant transcribes you, an open LLM generates the reply, and a text-to-speech model such as Kokoro speaks it back. The components communicate over a WebSocket server designed for low-latency streaming, and the pipeline handles interruptions, so you can talk over the agent and it will stop and listen, just like a human conversation. Every stage is swappable, which makes the repository both a working voice agent and a reference architecture for experimenting with open speech models, including on local hardware and in languages beyond English.
Speech-to-Speech is free and open source under the Apache-2.0 license. There are no usage fees; your costs are only the compute needed to run the models you select.
Speech-to-Speech is for developers and researchers who want to prototype or ship realtime voice assistants with open models, understand how the pieces of a voice pipeline fit together, or avoid dependence on closed voice APIs.