Vision Agents is Stream’s open-source Python framework for building real-time voice and video AI agents, published on GitHub with around 8,200 stars. It lets you plug any LLM, speech, or vision model into a live audio-video pipeline and ship agents for telehealth, voice support, live coaching, and similar real-time products.
Realtime agents are hard because audio, video, speech recognition, model calls, and speech synthesis all have to stay in sync at low latency. Vision Agents handles that plumbing. It offers two architecture modes: native Realtime APIs over WebRTC or WebSocket, or custom pipelines chaining speech-to-text, an LLM, and text-to-speech. More than 35 integrations are included — OpenAI, Gemini, Anthropic, Deepgram, ElevenLabs, YOLO and others — and video frames can be processed live with models like YOLO or Roboflow for detection and understanding. Telephony is supported through Twilio for voice calls, retrieval is built in via vector search integrations, and the framework targets production use with an HTTP server, Prometheus metrics, and Docker and Kubernetes deployment, running on Stream’s global edge network with sub-500ms latency. The project is under very active development, with updates continuing through late 2026.
The Vision Agents framework is free and open source under the Apache-2.0 license. Running agents in production typically uses Stream’s commercial edge and video platform, which has paid plans, so most real deployments treat it as freemium.
Vision Agents is for developers building realtime AI products — AI receptionists, telehealth assistants, live coaching, and video-monitoring agents — who want an open framework instead of gluing together media pipelines by hand.