Skip to main content

Prompt caching

Prompt caching lets a provider reuse the processed form of a long, stable prefix, such as a system prompt, a reference document or a set of tool definitions, across calls, at a fraction of the input price. The public API accepts cache marks in the Anthropic shape at three places in the request, forwards them to the providers that understand them, and strips them for every other provider. Nothing is translated.

Who can use it: anyone holding an API key with the Chat scope.

Where a mark can go​

PlaceShape
The top of the request"cache_control": {"type": "ephemeral"} beside messages
A message{"role": "system", "content": "...", "cache_control": {"type": "ephemeral"}}
A content blockInside a content list, {"type": "text", "text": "...", "cache_control": {"type": "ephemeral"}}

The value is not validated by the gateway. {"type": "ephemeral"} is the shape Anthropic accepts; anything else is forwarded as written and refused or ignored by the provider.

Which providers keep the marks​

The call is served byMarks
AnthropicKept
An Anthropic model on Bedrock or VertexKept
Any other providerStripped before the call, without an error

The decision is made per attempt, on the provider that actually serves the call. A tier can fall back from one provider to another between two calls, so the same request can be cached on one call and not on the next. Pin the model for cache-dependent workloads and read served_provider when a cache figure is missing.

Quick start​

Mark a long system prompt, send two questions against it, and read the cache figures:

curl -X POST "https://<your tenant host>/public/v1/chat/completions" \
-H "Authorization: Bearer kapi_..." \
-H "Content-Type: application/json" \
-d '{"model": "<an-anthropic-model-id>",
"messages": [
{"role": "system", "content": "<the full leave policy, several thousand words>", "cache_control": {"type": "ephemeral"}},
{"role": "user", "content": "How many days of carer leave?"}]}'

Read the usage​

FieldMeaning
usage.prompt_tokensEvery input token, cached or not
usage.cache_creation_input_tokensThe part of prompt_tokens written to the cache on this call
usage.cache_read_input_tokensThe part of prompt_tokens served from the cache
usage.cache_creation.ephemeral_5m_input_tokens, ephemeral_1h_input_tokensThe write split by cache lifetime, when the provider reports it

The fields are present only when the provider reported them; they are never stamped as zero. On a stream they arrive in the final usage frame. The platform records the same figures on the usage event and prices them: a cache read at a tenth of the input rate, a five-minute cache write at 1.25 times it and a one-hour write at twice it. A streamed call has no per-lifetime split, so its writes are priced at the five-minute rate.

What breaks a match​

  • A prefix the gateway rewrote. Two rewrites exist: guardrail_redaction, when an input rail redacted part of the prompt, and render_tool_filter, which applies to agent traffic only. The gateway discloses them in an X-Prefix-Modified header that the public edge does not relay on the current build. A rewritten prefix does not match what was cached.
  • Several system messages. They are merged into one leading system message before the provider sees them; the merged text is what is cached, so keep their order stable.
  • A change of provider through a tier fallback, as above.
  • The provider's own rules: minimum cacheable length and lifetime are theirs, not the gateway's.

Troubleshooting​

  • No cache fields in usage: the serving provider either strips the marks or reports nothing. Read served_provider; only Anthropic, and Anthropic models on Bedrock or Vertex, keep them.
  • cache_creation_input_tokens is set on every call and cache_read_input_tokens never: the prefix changes between calls. Check whether an input rail could have redacted the prompt, and make sure the marked text is byte-for-byte identical.
  • A 400 from the provider naming cache_control: the mark's shape is wrong; the gateway forwarded it as written.
  • The figures vanish after a while: the cache expired on the provider's schedule, and the next call is a write again.

The same call elsewhere

Describes the platform as of .

Was this page helpful?