NeMo Switchyard is NVIDIA’s free, open-source routing layer for large language model applications. It sits between your application and model providers, routing traffic across models and providers while preserving native OpenAI and Anthropic API compatibility — so you can switch models, benchmark alternatives and optimise the cost/performance trade-off without rewriting your client code.
Switchyard is an open-source project on GitHub from the NVIDIA NeMo team, with about 3,400 stars and active development in 2026. LLM applications are usually hard-wired to one provider’s API; Switchyard removes that coupling by presenting a single compatible endpoint and distributing requests according to routing policies you define. That makes it practical to A/B test a new model against your production model, fail over between providers, or send simple requests to cheaper models and hard ones to frontier models. Because it preserves the native OpenAI and Anthropic API shapes, existing SDKs and tools keep working unchanged — you just point them at Switchyard.
NeMo Switchyard is completely free and open source under the Apache-2.0 license. You self-host the router and only pay your model providers for the API calls it forwards.
Switchyard is for platform and application teams running LLM features in production who want provider independence, systematic model benchmarking and fine-grained control over cost versus quality.