# Structured outputs

Ask a model for JSON with response_format, find out which models honour it, and validate the result yourself, because the gateway forwards the request and checks nothing.

`response_format` takes the OpenAI shapes, `{"type": "json_object"}` and `{"type": "json_schema", "json_schema": {...}}`, and the gateway forwards it to the provider exactly as sent. It does not translate between providers, validate the answer against your schema, or repair broken JSON, and for a model that does not support the field it drops it without a word. So there are two jobs on your side: pick a model that supports it, and validate what comes back.

**Who can use it:** anyone holding an API key with the **Chat** scope, and **Read models** to check a model's capabilities first.

## Which models honour it

The models route reports a `capabilities` object per model. Its `json` flag comes from a static table of what each model supports, not from a live check, and is left out when the table does not know.

| `capabilities.json` | What to do |
| --- | --- |
| `true` | Send `response_format`, with a schema if you have one, and validate the result. |
| `false` | The field will be dropped. Ask for JSON in the prompt and validate harder. |
| absent | Unknown. Try it and validate; treat prose as the field having been dropped. |

Pin the `model` for anything that depends on this. A tier can fall back to another model on the next call, and the fallback may not support the field; `served_model` says which one answered.

## Quick start

Ask for a fixed shape and check it before use:

**cURL**

```bash
curl -X POST "https://<your tenant host>/public/v1/chat/completions" \
  -H "Authorization: Bearer kapi_..." \
  -H "Content-Type: application/json" \
  -d '{"model": "<your-model-id>",
       "messages": [{"role": "user", "content": "Extract the fields from: Invoice 4471 from Northwind, total 1,280.50 EUR, due 30 September 2026."}],
       "response_format": {"type": "json_schema", "json_schema": {"name": "invoice", "schema": {
         "type": "object", "properties": {"number": {"type": "string"}, "supplier": {"type": "string"},
         "total": {"type": "number"}, "currency": {"type": "string"}, "due": {"type": "string"}},
         "required": ["number", "supplier", "total", "currency", "due"], "additionalProperties": false}}}}'
```

**Python**

```python

BASE = "https://<your tenant host>/public/v1"
HEADERS = {"Authorization": "Bearer kapi_..."}
SCHEMA = {"type": "object", "additionalProperties": False,
          "properties": {"number": {"type": "string"}, "supplier": {"type": "string"}, "total": {"type": "number"},
                         "currency": {"type": "string"}, "due": {"type": "string"}},
          "required": ["number", "supplier", "total", "currency", "due"]}

r = requests.post(f"{BASE}/chat/completions", headers=HEADERS, json={
    "model": "<your-model-id>",
    "messages": [{"role": "user", "content": "Extract the fields from: Invoice 4471 from Northwind, total 1,280.50 EUR, due 30 September 2026."}],
    "response_format": {"type": "json_schema", "json_schema": {"name": "invoice", "schema": SCHEMA}},
}).json()

choice = r["choices"][0]
if choice["finish_reason"] == "content_filter":
    raise RuntimeError(f"blocked by guardrails: {r.get('guardrail')}")
invoice = json.loads(choice["message"]["content"])           # raises on prose or broken JSON
missing = [k for k in SCHEMA["required"] if k not in invoice]
if missing:
    raise ValueError(f"model answered without {missing}; served by {r['served_model']}")
print(invoice)
```

**TypeScript**

```typescript
const BASE = "https://<your tenant host>/public/v1";
const headers = { Authorization: "Bearer kapi_...", "Content-Type": "application/json" };
const schema = { type: "object", additionalProperties: false,
  properties: { number: { type: "string" }, supplier: { type: "string" }, total: { type: "number" },
                currency: { type: "string" }, due: { type: "string" } },
  required: ["number", "supplier", "total", "currency", "due"] };

const r = await (await fetch(`${BASE}/chat/completions`, {
  method: "POST", headers,
  body: JSON.stringify({
    model: "<your-model-id>",
    messages: [{ role: "user", content: "Extract the fields from: Invoice 4471 from Northwind, total 1,280.50 EUR, due 30 September 2026." }],
    response_format: { type: "json_schema", json_schema: { name: "invoice", schema } },
  }),
})).json();

const choice = r.choices[0];
if (choice.finish_reason === "content_filter") throw new Error(`blocked by guardrails: ${JSON.stringify(r.guardrail)}`);
const invoice = JSON.parse(choice.message.content);           // throws on prose or broken JSON
const missing = schema.required.filter((k) => !(k in invoice));
if (missing.length) throw new Error(`model answered without ${missing.join(", ")}; served by ${r.served_model}`);
console.log(invoice);
```

## Limits and behaviour

- The gateway makes no distinction between `json_object` and `json_schema`; whatever the provider does with each is what you get. A `strict` flag inside `json_schema` reaches the provider unchanged and is enforced there or not at all.
- Check `finish_reason` before parsing. An output guardrail that blocks replaces the content with a banner and sets `finish_reason` to `content_filter`; a redaction swaps text inside the JSON, which can break it. Both are announced in the `guardrail` object.
- A refusal counts as an answer, not as an empty response: a model that declines to fill the schema returns a `refusal` and does not trigger a fallback to another model.
- Tools and `response_format` together follow the tool rules: on a first turn with tools bound the model is made to call a tool, so the JSON answer arrives on the turn after the tool result. See [Call tools from a completion](/developers/guides/call-tools-from-a-completion).
- Streaming buffers the whole answer before replaying it, so a JSON answer never arrives half-formed mid-stream; it arrives whole, in one burst, then `[DONE]`.

## Troubleshooting

- **The answer is prose, not JSON**: the served model does not support the field and it was dropped. Read `served_model`, check its `capabilities.json`, and pin a model that has it.
- **The JSON does not match the schema**: nothing in the gateway enforces it. Validate, and on failure send the answer back with the validation error as the next user message.
- **`finish_reason` is `content_filter`**: a guardrail blocked the answer; the `guardrail` object names the rail. This is a policy outcome, not a formatting one.
- **The same request works on one call and not the next**: a tier fallback served the second call. Pin the model.

## Related

- [Tiers and models](/developers/guides/tiers-and-models)
- [Call tools from a completion](/developers/guides/call-tools-from-a-completion)
- [Use an OpenAI client library](/developers/guides/use-an-openai-client-library)
- Tools and models

Source: https://docs.katonic.ai/developers/guides/structured-outputs
