Multi-provider model routing with same-model failover

Model routing now lives in one engine, and a model can be served by more than one route. Every chat model in the catalog has its native provider plus routes through Vercel AI Gateway and OpenRouter. When a provider is overloaded, a run first moves the same model to its next untried route, and only then falls back to a different model, so a provider incident doesn't have to change which model is doing your work.

How it works

• Ranked routes. Native lab APIs (Anthropic, Vertex, OpenAI, xAI, Fireworks) first, then Vercel AI Gateway, then OpenRouter.
• Route hops are free. A run tracks the routes and the models it has tried separately, so switching routes doesn't use up a model fallback.
• Caching follows the route. Each route declares how it handles prompt-cache breakpoints, reasoning effort, and session affinity, so a Claude request through a gateway keeps its cache discounts.
• Embeddings too. Embedding requests use the same routes, and a route returning vectors of the wrong size is rejected so stored vectors stay comparable.

How we verified it

Live suites run every route of every model at every effort level, plus streaming with tools, vision, and cache reads. They caught a provider alias that had started returning errors in production, which is now pinned to its dated version.

Also shipped

• Muse Spark 1.3, the first model served only through a gateway.