# Stream a reply

What each streaming route sends, event by event, for a model completion and for an agent turn, how a failure arrives mid-stream, and what a strict client needs to read both.

Two public routes stream, and they stream differently. Chat completions sends OpenAI chunk events as the model writes them, unless your organisation's guardrail profile must read the whole answer first. The agent chat route relays the agent's run as it happens, as typed events that cover text, tool calls, steps and receipts. Both use server-sent events with `data:` lines only and end with `data: [DONE]`.

**Who can use it:** anyone holding an API key with the **Chat** scope; text completions stream the same way as chat completions under the **Completions** scope.

## Which stream you get

| | `POST /public/v1/chat/completions` with `stream` | `POST /public/v1/chat` with `stream` |
| --- | --- | --- |
| Timing | As the model writes; generated, checked, then sent at once when the guardrail profile has output rails | Relayed as the agent produces it |
| Event shape | OpenAI `chat.completion.chunk` objects | Objects with a `type` field |
| Text | `choices[0].delta.content` | `TEXT_MESSAGE_CONTENT` events with `delta` |
| A failure after the start | One chunk with `finish_reason` set to `error` and an `error` object, then `[DONE]`; status stays 200 | A `RUN_ERROR` event with `error`, `error_code` and `retryable`; status stays 200 |
| Usage | A late chunk, usually the last, with a `usage` object; its `choices` holds one entry with an empty delta | `RUN_FINISHED` with a `usage` object |
| Heartbeats | None | None |
| Does not stream | | `POST /public/v1/<namespace>/<slug>/chat` ignores `stream` |

## Read a completion stream

Each event is a chunk as the provider shaped it, stamped with `model`, `served_model` and `served_provider`, plus `is_fallback`, `fallback_from` and `fallback_reason` on the first when the tier's primary did not serve. A thinking model (the Qwen3, QwQ and DeepSeek-R1 families, not their instruct or coder variants) streams its reasoning live on `delta.reasoning_content` from its first token, then the answer on `delta.content`; the `</think>` tag never appears in the answer. Tool calls arrive as `delta.tool_calls` with `index` and `function` filled in. The last data chunk before `[DONE]` has empty `choices` and the `usage` object; the gateway asks every provider that can send it and synthesises one for self-hosted engines. Anthropic and Bedrock are not asked, so it can be absent there.

**cURL**

```bash
curl -N -X POST "https://<your tenant host>/public/v1/chat/completions" \
  -H "Authorization: Bearer kapi_..." \
  -H "Content-Type: application/json" \
  -d '{"tier": "fast", "stream": true, "messages": [{"role": "user", "content": "Three tips for a first day at work."}]}'
```

**Python**

```python

BASE = "https://<your tenant host>/public/v1"
HEADERS = {"Authorization": "Bearer kapi_..."}

with requests.post(f"{BASE}/chat/completions", headers=HEADERS, stream=True, json={
    "tier": "fast", "stream": True,
    "messages": [{"role": "user", "content": "Three tips for a first day at work."}],
}) as r:
    r.raise_for_status()
    for line in r.iter_lines():
        if not line.startswith(b"data: "):
            continue
        payload = line[6:]
        if payload == b"[DONE]":
            break
        chunk = json.loads(payload)
        if "error" in chunk:                                   # failure after the start; status was 200
            raise RuntimeError(chunk["error"]["message"])
        if chunk.get("usage"):
            print("\nusage:", chunk["usage"], "served by", chunk["served_model"])
        for choice in chunk.get("choices", []):
            print(choice["delta"].get("content") or "", end="", flush=True)
```

**TypeScript**

```typescript
const BASE = "https://<your tenant host>/public/v1";
const headers = { Authorization: "Bearer kapi_...", "Content-Type": "application/json" };

const r = await fetch(`${BASE}/chat/completions`, {
  method: "POST", headers,
  body: JSON.stringify({ tier: "fast", stream: true, messages: [{ role: "user", content: "Three tips for a first day at work." }] }),
});
if (!r.ok || !r.body) throw new Error(`${r.status} ${await r.text()}`);
const reader = r.body.pipeThrough(new TextDecoderStream()).getReader();
let buffer = "";
for (;;) {
  const { value, done } = await reader.read();
  if (done) break;
  buffer += value;
  let nl: number;
  while ((nl = buffer.indexOf("\n\n")) >= 0) {
    const frame = buffer.slice(0, nl); buffer = buffer.slice(nl + 2);
    if (!frame.startsWith("data: ")) continue;
    const payload = frame.slice(6);
    if (payload === "[DONE]") break;
    const chunk = JSON.parse(payload);
    if (chunk.error) throw new Error(chunk.error.message);            // failure after the start; status was 200
    if (chunk.usage) console.log("\nusage:", chunk.usage, "served by", chunk.served_model);
    for (const c of chunk.choices ?? []) process.stdout.write(c.delta?.content ?? "");
  }
}
```

A failure after the start is a chunk with `finish_reason` `error` and an `error` with `message` and `type`, plus `code` `STREAM_ERROR` when the gateway failed, then `[DONE]`. Before the first token, reasoning included, the gateway can still move to another model in the tier; after it, a failure ends the stream.

### When the answer is held

Chunks leave as written only when the guardrail profile has no output rails: no output checks and no secret redaction. Otherwise, or when the profile cannot be read, the gateway reads the whole answer, redacts or blocks it, and sends the chunks together. On AI Cloud each organisation's profile decides.

An input rail that blocks the question sends one chunk with the refusal text, `finish_reason` `content_filter` and a `guardrail` object with `direction` `input`, then `[DONE]`. A zero usage chunk follows only if you asked for usage in `stream_options`. `guardrail.rails` never includes the matched value.

## Read an agent stream

Every event carries `type` and `timestamp`; fields that are not set are absent. A normal turn runs `RUN_STARTED`, then text and tool events, then `STEP_STARTED` and `STEP_FINISHED` pairs, then `RUN_FINISHED`, then `[DONE]`.

| Event | Fields | Meaning |
| --- | --- | --- |
| `RUN_STARTED` | `run_id`, `thread_id` | The turn was admitted; `thread_id` is the session's canonical id |
| `TEXT_MESSAGE_START`, `TEXT_MESSAGE_CONTENT`, `TEXT_MESSAGE_END` | `message_id`, `role`; `delta` | The reply, in word-sized deltas |
| `TOOL_CALL_START`, `TOOL_CALL_ARGS`, `TOOL_CALL_RESULT`, `TOOL_CALL_END` | `tool_call_id`, `tool_call_name`; `args` as a string; `result` as a string with `is_error`, `truncated` and `duration_ms` | A tool the agent used |
| `STEP_STARTED`, `STEP_FINISHED` | `step_name`, `step_index`, `step_type`, `tokens_input`, `tokens_output`, `step_latency_ms` | One model step each, after the text |
| `CUSTOM` | `name`, `value` | `sources` (the citations), `knowledge_retrieval`, `guardrail_block`, `guardrail_redaction`, `guardrail_defense`, `receipt`, `ui.block_limit` |
| `STATE_DELTA`, `ARTIFACT_CREATED`, `ARTIFACT_UPDATED`, `ACTIVITY_SNAPSHOT` | `state`; `name` and `value`; `activity` | Agents that keep state or write artifacts |
| `RUN_FINISHED` | `usage` with `prompt_tokens`, `completion_tokens`, `total_tokens`, `model`, `cost`, `estimated` | The turn succeeded |
| `RUN_ERROR` | `error`, `error_code`, `retryable`, and `run_id` and `thread_id` when known | The turn failed. `error_code` is one of `provider_failure`, `gateway`, `aborted`, `agent_not_found`, `model_not_vision_capable`, `agent_type_cannot_read_images` or `unknown` |

**cURL**

```bash
curl -N -X POST "https://<your tenant host>/public/v1/chat" \
  -H "Authorization: Bearer kapi_..." \
  -H "Content-Type: application/json" \
  -d '{"agent_id": "support-assistant", "session_id": "<session-id>", "stream": true,
       "messages": [{"role": "user", "content": "And for annual plans?"}]}'
```

**Python**

```python
with requests.post(f"{BASE}/chat", headers=HEADERS, stream=True, json={
    "agent_id": "support-assistant", "session_id": "<session-id>", "stream": True,
    "messages": [{"role": "user", "content": "And for annual plans?"}],
}) as r:
    r.raise_for_status()
    for line in r.iter_lines():                                # read to the end, not to [DONE]
        if not line.startswith(b"data: ") or line == b"data: [DONE]":
            continue
        event = json.loads(line[6:])
        kind = event["type"]
        if kind == "TEXT_MESSAGE_CONTENT":
            print(event["delta"], end="", flush=True)
        elif kind == "CUSTOM" and event["name"] == "sources":
            print("\nsources:", event["value"])
        elif kind == "RUN_FINISHED":
            print("\nusage:", event.get("usage"))
        elif kind == "RUN_ERROR":
            raise RuntimeError(f"{event['error_code']}: {event['error']} (retryable={event.get('retryable')})")
```

**TypeScript**

```typescript
const r = await fetch(`${BASE}/chat`, {
  method: "POST", headers,
  body: JSON.stringify({ agent_id: "support-assistant", session_id: "<session-id>", stream: true,
                         messages: [{ role: "user", content: "And for annual plans?" }] }),
});
if (!r.ok || !r.body) throw new Error(`${r.status} ${await r.text()}`);
const reader = r.body.pipeThrough(new TextDecoderStream()).getReader();
let buffer = "";
for (;;) {                                                     // read to the end, not to [DONE]
  const { value, done } = await reader.read();
  if (done) break;
  buffer += value;
  let nl: number;
  while ((nl = buffer.indexOf("\n\n")) >= 0) {
    const frame = buffer.slice(0, nl); buffer = buffer.slice(nl + 2);
    if (!frame.startsWith("data: ") || frame === "data: [DONE]") continue;
    const event = JSON.parse(frame.slice(6));
    if (event.type === "TEXT_MESSAGE_CONTENT") process.stdout.write(event.delta);
    else if (event.type === "CUSTOM" && event.name === "sources") console.log("\nsources:", event.value);
    else if (event.type === "RUN_FINISHED") console.log("\nusage:", event.usage);
    else if (event.type === "RUN_ERROR") throw new Error(`${event.error_code}: ${event.error}`);
  }
}
```

## What a strict client needs

- Only `data:` lines, separated by a blank line. There are no `event:` or `id:` fields, and `[DONE]` is not JSON.
- No heartbeats on either route, so set the read timeout by the route. A completion stream held by output rails is silent until the whole answer is ready; the gateway allows up to fifteen seconds to the first token for chat models, longer for reasoning models, and two minutes of idle after that. An agent stream can pause during a tool call, and the agent's own limit on a turn is about five minutes.
- The status is 200 once the stream has started. Treat a chunk with `finish_reason` set to `error`, or a `RUN_ERROR` event, as the failure.
- On the agent route, read until the connection closes, not to `[DONE]`: an answer that produced reasoning and no text sends a `RUN_ERROR` after the terminal frame. A failure before the run is admitted is a `RUN_ERROR` with no `RUN_STARTED`.
- On the agent route an input refusal streams as ordinary text with zero usage, and an output block arrives as a `CUSTOM` event named `guardrail_block` with the replacement text.
- A service that stops sending for five minutes is cut without an error event. Treat a stream that ends without `RUN_FINISHED` or `[DONE]` as failed.

## Troubleshooting

- **Every completion chunk arrives at the same moment**: the guardrail profile has output rails, so the gateway checks the whole answer before sending it. That is by design.
- **A chunk carries `error` and the request returned 200**: the provider failed after the stream started. Retry the call.
- **`RUN_ERROR` with `agent_not_found`**: the `agent_id` does not exist; the buffered form of the same call answers 404.
- **`RUN_ERROR` with `gateway` after a long pause**: the agent's turn ran out of time. Shorten the question or the tools involved.
- **No events at all from the registry-path route**: it ignores `stream`. Use `POST /public/v1/chat` with `agent_id` for a stream.

## Related

- [Agent chat or model completion](/developers/guides/agent-chat-or-model-completion)
- [Chat from your application](/developers/guides/chat-from-your-application)
- Streaming events
- Chat and completions

Source: https://docs.katonic.ai/developers/guides/stream-a-reply
