Caching

Opt-in per workspace. Disabled by default.

Enable

Workspace → Cache → toggle on. Default TTL: 1 hour.

Cache key

SHA-256 of the normalized request body — model, messages (or rendered prompt), temperature, top_p, max_tokens, tools, response_format, etc. Anything that changes the output changes the key.

Headers (including x-airouter-end-user) are not part of the key — a cache hit for user A serves user B if the body matches.

Behavior

Hit → response returned instantly, x-airouter-cache: hit, x-airouter-cost-usd: 0, vendor not called.
Miss → upstream call as normal, response stored, x-airouter-cache: miss.
Disabled → x-airouter-cache: off.

When it helps

Deterministic prompts (FAQ answering, classification).
Retry-heavy flows where the same body lands twice.
Demo / load-test traffic.

When to skip

Per-user personalization in the prompt body (cache will be useless or worse, leaky — but only if you put PII in the body; the header doesn't affect the key).
Tool-using agents where state changes.

Invalidation

Cache expires on TTL. No manual purge today — adjust the prompt or version to change the cache key.

What it saved

Workspace → Cache shows spend avoided, hit rate and tokens not re-billed for the last 30 days, broken down per calling function. Organisation admins see the same figures per workspace under Organisation → Finance. Both read the request ledger (cache hits are logged with cache: hit and cached_tokens), priced at the model's own rate — so the numbers cover past history, not just from the day counting was switched on.

Two caches, two switches

	odnoga cache	Provider prompt cache
What happens	we return an answer we already stored; no provider call	the provider re-reads prompt text it holds, at a lower rate
Can it change an answer	yes, for sampled calls — that is why safety mode exists	never; the answer is always generated fresh
Saving	the whole call	the repeated prompt tokens
Setting	`cache_enabled` + `cache_mode` + `cache_ttl_seconds` + `cache_scope`	`vendor_prompt_cache_enabled`

cache_mode = safe (default) reuses an answer only for repeatable calls (seed set, or temperature = 0 with top_p unset/0/1). always reuses any identical call and must be an explicit choice.
cache_scope = end_user puts the end-user id in the cache key, so one customer never receives another's answer.
Per-function and per-prompt overrides are set in Settings → Caching; a prompt rule beats a function rule, and a disabled rule opts that path out even when the workspace cache is on.
Turning vendor_prompt_cache_enabled off strips cache_control markers from messages, system and tools before the request leaves us.

Per-call override

x-odnoga-cache: bypass (ignore any stored answer), refresh (deliberately miss and re-store) or off. A reused answer carries x-odnoga-cache-age and _odnoga.cache_age_s / _odnoga.cached_at, so nothing is silently stale. Skips are reported via x-odnoga-cache-skipped.

Settings over MCP

cache.settings.get, cache.settings.set (routing:read / routing:write) and cache.stats (usage:read).