Prompt-cache-aware pricing for every AI model

Agent runs resend the same long prefix on every step: instructions, tool definitions, and the conversation so far. Most of it comes back from the provider's prompt cache at a fraction of the input price. Every model in the catalog now carries four rates (input, output, cache read, cache write), and each call is priced on what the provider actually served.

How it works

• Three input buckets. Uncached tokens bill at the input rate, cache reads at the cache-read rate, and cache writes at the cache-write rate. On Claude Sonnet 5, for example, input is $3.00 per million tokens and a cache read is $0.30.
• More cache hits. Each conversation's requests carry a session-affinity key, so providers serving open-weight models route a conversation back to the replica holding its warm cache. For Anthropic models we mark the end of the history as a cache breakpoint, so earlier turns read from cache on every step.
• Pruning only when needed. Old tool results are trimmed only when a conversation nears its context limit, since pruning earlier would invalidate the cache.
• Auditable. Every usage record stores its cache-read and cache-write counts.

Also shipped

• Credits deduct while a run is in progress, and open dashboards update the balance live.
• Workflows that keep failing because credits ran out now pause with a notice saying how to resume.