# Use an OpenAI client library

Point an OpenAI client library at your tenant's public API to list models, run chat completions, receive a streamed reply and attribute usage to an agent.

The model routes of the public API accept the request shapes OpenAI client libraries already send. Change two settings, the base URL and the key, and the official libraries work against your tenant.

**Who can use it:** anyone holding an API key with the **Chat** scope for chat completions and the **Read models** scope to list models; the other model routes name their scope on their reference page. The tenant must be licensed for the public API.

## Prerequisites

- An API key. The **Chat only** preset covers chat completions; add **Read models** if the application lists models itself. See Authentication.
- The `openai` package for Python or Node.js, or any client that lets you set a base URL.

## Configure the library

1. **Find a model id.** The models route returns the models the AI Gateway serves for your tenant in the OpenAI list shape, chat models by default.

   ```bash
   curl "https://<your tenant host>/public/v1/models" \
     -H "Authorization: Bearer kapi_..."
   ```

2. **Point the library at your tenant.** The base URL is `https://<your tenant host>/public/v1` and the key is the bearer token.

   ```python
   from openai import OpenAI

   client = OpenAI(base_url="https://<your tenant host>/public/v1", api_key="kapi_...")

   completion = client.chat.completions.create(
       model="<your-model-id>",
       messages=[{"role": "user", "content": "Summarise our onboarding guide in three bullet points."}],
   )
   print(completion.choices[0].message.content)
   ```

   ```javascript
   import OpenAI from "openai";

   const client = new OpenAI({ baseURL: "https://<your tenant host>/public/v1", apiKey: "kapi_..." });

   const completion = await client.chat.completions.create({
     model: "<your-model-id>",
     messages: [{ role: "user", content: "Summarise our onboarding guide in three bullet points." }],
   });
   console.log(completion.choices[0].message.content);
   ```

3. **Let the tier choose, if you prefer.** Called directly, the route accepts a request without `model` and routes by `tier` (`balanced` unless you say otherwise). The client libraries require a model argument, so pass a model id from step 1 when you go through them.

## Stream tokens

Set `stream` to true and the route answers with a text/event-stream: each event is a `data:` line carrying a chat completion chunk as JSON, and the stream ends with `data: [DONE]`. The libraries iterate the chunks for you.

The chunks arrive as the model produces them, so the loop below prints the reply word by word. The exception is a guardrail profile with output rails (output checks or secret redaction): the gateway then holds the whole answer so those rails can read it, and the chunks arrive in one burst. [Stream a reply](/developers/guides/stream-a-reply) has the event catalogue for both the model routes and the agent routes.

```python
with client.chat.completions.create(
    model="<your-model-id>",
    messages=[{"role": "user", "content": "Write a short welcome note for new starters."}],
    stream=True,
) as stream:
    for chunk in stream:
        delta = chunk.choices[0].delta.content or ""
        print(delta, end="", flush=True)
```

Generation routes have a longer time budget than the rest of the API, so a long answer is not cut off at the sixty-second mark that applies elsewhere.

## Attribute usage to an agent

Send an `X-Agent-ID` header carrying an agent id and the gateway records the usage against that agent in analytics.

```python
client = OpenAI(
    base_url="https://<your tenant host>/public/v1",
    api_key="kapi_...",
    default_headers={"X-Agent-ID": "<agent-id>"},
)
```

## Tools, JSON output and provider fields

`tools`, `tool_choice` and `response_format` take the OpenAI shapes, with three differences from a direct provider call. On a first turn with tools bound, an absent or `auto` tool choice is upgraded to `required`, so the model must call a tool; `none` and a named tool are respected, and the requirement is relaxed on later turns. `response_format` is forwarded as sent, and dropped without an error for a model that does not support it. `extra_body` is not a passthrough: only ten sampling parameters for self-hosted engines are kept (`min_tokens`, `ignore_eos`, `top_k`, `min_p`, `repetition_penalty`, `length_penalty`, `stop_token_ids`, `include_stop_str_in_output`, `skip_special_tokens`, `spaces_between_special_tokens`) and every other key is dropped.

`user` and `metadata` are accepted and discarded: neither reaches the model, the usage record or the logs, so attribute usage with the `X-Agent-ID` header above. The other model routes, completions, embeddings, rerank and image generation, are listed on the Chat and completions page with the scope each one needs; [Call tools from a completion](/developers/guides/call-tools-from-a-completion) and [Structured outputs](/developers/guides/structured-outputs) go deeper.

## Verify

The response is an OpenAI-compatible chat completion object with `choices[0].message.content`, and the key's row on the Developer Hub's **API keys** tab shows the usage.

## Troubleshooting

- **401 with `AUTH_ERROR`**: check that the library sends the key as the bearer token and that the base URL ends in `/public/v1`.
- **401 with `AUTH_ERROR` and `Invalid or expired API key`, but the key works elsewhere**: the key lacks the scope of the route you called, and the gateway reports a missing scope as an invalid key. Chat completions need Chat; the models list needs Read models.
- **429 with `RATE_LIMITED`**: the key is over its own per-minute or per-day limit; read `X-RateLimit-Reset` and retry after it. See API overview.
- **429 with `error.code` set to `rate_limit_exceeded`**: a per-user, per-tenant or per-model limit inside the AI Gateway; the gateway's `Retry-After` is not relayed, so wait for the next clock minute. See [How limits combine](/developers/guides/how-limits-combine).
- **502 or 504**: the gateway could not get an answer from the provider in time. Retry with backoff; if it persists, an administrator can check the provider in the Control Room's AI Gateway.

## Related

- Chat and completions
- Tools and models
- [Chat from your application](/developers/guides/chat-from-your-application)
- Errors
- [Integrations](/developers/integrations/overview), one verified page per client library and front end

Source: https://docs.katonic.ai/developers/guides/use-an-openai-client-library
