# Prompt caching

Mark a stable prefix for caching with cache_control, know which providers keep the marks and which strip them, and read the cache figures the usage object reports.

Prompt caching lets a provider reuse the processed form of a long, stable prefix, such as a system prompt, a reference document or a set of tool definitions, across calls, at a fraction of the input price. The public API accepts cache marks in the Anthropic shape at three places in the request, forwards them to the providers that understand them, and strips them for every other provider. Nothing is translated.

**Who can use it:** anyone holding an API key with the **Chat** scope.

## Where a mark can go

| Place | Shape |
| --- | --- |
| The top of the request | `"cache_control": {"type": "ephemeral"}` beside `messages` |
| A message | `{"role": "system", "content": "...", "cache_control": {"type": "ephemeral"}}` |
| A content block | Inside a content list, `{"type": "text", "text": "...", "cache_control": {"type": "ephemeral"}}` |

The value is not validated by the gateway. `{"type": "ephemeral"}` is the shape Anthropic accepts; anything else is forwarded as written and refused or ignored by the provider.

## Which providers keep the marks

| The call is served by | Marks |
| --- | --- |
| Anthropic | Kept |
| An Anthropic model on Bedrock or Vertex | Kept |
| Any other provider | Stripped before the call, without an error |

The decision is made per attempt, on the provider that actually serves the call. A tier can fall back from one provider to another between two calls, so the same request can be cached on one call and not on the next. Pin the `model` for cache-dependent workloads and read `served_provider` when a cache figure is missing.

## Quick start

Mark a long system prompt, send two questions against it, and read the cache figures:

**cURL**

```bash
curl -X POST "https://<your tenant host>/public/v1/chat/completions" \
  -H "Authorization: Bearer kapi_..." \
  -H "Content-Type: application/json" \
  -d '{"model": "<an-anthropic-model-id>",
       "messages": [
         {"role": "system", "content": "<the full leave policy, several thousand words>", "cache_control": {"type": "ephemeral"}},
         {"role": "user", "content": "How many days of carer leave?"}]}'
```

**Python**

```python

BASE = "https://<your tenant host>/public/v1"
HEADERS = {"Authorization": "Bearer kapi_..."}
POLICY = open("leave-policy.txt").read()          # several thousand words, identical on every call

def ask(question):
    r = requests.post(f"{BASE}/chat/completions", headers=HEADERS, json={
        "model": "<an-anthropic-model-id>",
        "messages": [
            {"role": "system", "content": POLICY, "cache_control": {"type": "ephemeral"}},
            {"role": "user", "content": question},
        ],
    }).json()
    u = r["usage"]
    print(f"{r['served_provider']}: prompt={u['prompt_tokens']} "
          f"written={u.get('cache_creation_input_tokens', 'n/a')} read={u.get('cache_read_input_tokens', 'n/a')}")
    return r["choices"][0]["message"]["content"]

ask("How many days of carer leave?")       # expect a cache write
ask("Does it accrue for part-time staff?") # expect a cache read
```

**TypeScript**

```typescript

const BASE = "https://<your tenant host>/public/v1";
const headers = { Authorization: "Bearer kapi_...", "Content-Type": "application/json" };
const policy = await readFile("leave-policy.txt", "utf8");   // identical on every call

async function ask(question: string) {
  const r = await (await fetch(`${BASE}/chat/completions`, {
    method: "POST", headers,
    body: JSON.stringify({
      model: "<an-anthropic-model-id>",
      messages: [
        { role: "system", content: policy, cache_control: { type: "ephemeral" } },
        { role: "user", content: question },
      ],
    }),
  })).json();
  const u = r.usage;
  console.log(`${r.served_provider}: prompt=${u.prompt_tokens} written=${u.cache_creation_input_tokens ?? "n/a"} read=${u.cache_read_input_tokens ?? "n/a"}`);
  return r.choices[0].message.content;
}

await ask("How many days of carer leave?");        // expect a cache write
await ask("Does it accrue for part-time staff?");  // expect a cache read
```

## Read the usage

| Field | Meaning |
| --- | --- |
| `usage.prompt_tokens` | Every input token, cached or not |
| `usage.cache_creation_input_tokens` | The part of `prompt_tokens` written to the cache on this call |
| `usage.cache_read_input_tokens` | The part of `prompt_tokens` served from the cache |
| `usage.cache_creation.ephemeral_5m_input_tokens`, `ephemeral_1h_input_tokens` | The write split by cache lifetime, when the provider reports it |

The fields are present only when the provider reported them; they are never stamped as zero. On a stream they arrive in the final usage frame. The platform records the same figures on the usage event and prices them: a cache read at a tenth of the input rate, a five-minute cache write at 1.25 times it and a one-hour write at twice it. A streamed call has no per-lifetime split, so its writes are priced at the five-minute rate.

## What breaks a match

- A prefix the gateway rewrote. Two rewrites exist: `guardrail_redaction`, when an input rail redacted part of the prompt, and `render_tool_filter`, which applies to agent traffic only. The gateway discloses them in an `X-Prefix-Modified` header that the public edge does not relay on the current build. A rewritten prefix does not match what was cached.
- Several `system` messages. They are merged into one leading system message before the provider sees them; the merged text is what is cached, so keep their order stable.
- A change of provider through a tier fallback, as above.
- The provider's own rules: minimum cacheable length and lifetime are theirs, not the gateway's.

## Troubleshooting

- **No cache fields in `usage`**: the serving provider either strips the marks or reports nothing. Read `served_provider`; only Anthropic, and Anthropic models on Bedrock or Vertex, keep them.
- **`cache_creation_input_tokens` is set on every call and `cache_read_input_tokens` never**: the prefix changes between calls. Check whether an input rail could have redacted the prompt, and make sure the marked text is byte-for-byte identical.
- **A 400 from the provider naming `cache_control`**: the mark's shape is wrong; the gateway forwarded it as written.
- **The figures vanish after a while**: the cache expired on the provider's schedule, and the next call is a write again.

## Related

- [Tiers and models](/developers/guides/tiers-and-models)
- [Attribute usage](/developers/guides/attribute-usage)
- [Use an OpenAI client library](/developers/guides/use-an-openai-client-library)
- Chat and completions

Source: https://docs.katonic.ai/developers/guides/prompt-caching
