# OpenAI Python SDK

Point the official openai package at your tenant's public API, list the models it serves, run a chat completion, stream a reply and read the usage that comes back.

The `openai` package is the client most Python code already uses for chat completions. Two constructor arguments, a base URL and a key, point it at your tenant's public API. From then on `models.list()` returns the models your tenant serves, `chat.completions.create()` runs through your tenant's AI Gateway with its tiers, guardrails, budgets and logs, and every call is recorded against your key.

**Who can use it:** anyone holding an API key with the **Chat** scope; add **Read models** if the application lists models itself. The tenant must be licensed for the public API.

## Prerequisites

- Python 3.9 or later and `pip install openai`. Verified with `openai` 2.54.0.
- Your tenant's public API address, `https://<your tenant host>/public/v1`. Studio's **Developer Hub** shows it at the top of the page.
- A key from the Developer Hub's **API keys** tab. The **Chat only** preset is enough to complete; add **Read models** to list models.

## Connect

```python
from openai import OpenAI

client = OpenAI(
    base_url="https://<your tenant host>/public/v1",
    api_key="kapi_YOUR_KEY",
)

models = client.models.list()
print([m.id for m in models.data][:5])

completion = client.chat.completions.create(
    model="<a model id from the list>",
    messages=[{"role": "user", "content": "Summarise our onboarding guide in three bullet points."}],
    max_tokens=300,
)
print(completion.model, completion.usage.completion_tokens)
print(completion.choices[0].message.content)
```

The library needs a `model` argument. The models route lists the chat models the AI Gateway serves for your tenant, and [Tiers and models](/developers/guides/tiers-and-models) explains how a model id relates to a tier. Keep the key out of source: the library reads `OPENAI_API_KEY` and `OPENAI_BASE_URL` from the environment when the arguments are omitted, so a shell that exports both runs unchanged code against your tenant.

To record the usage against one of your agents in analytics, add `default_headers={"X-Agent-ID": "<agent id>"}` to the constructor.

## Stream a reply

```python
with client.chat.completions.create(
    model="<a model id from the list>",
    messages=[{"role": "user", "content": "Write a short welcome note for new starters."}],
    max_tokens=300,
    stream=True,
    stream_options={"include_usage": True},
) as stream:
    for chunk in stream:
        if chunk.choices and chunk.choices[0].delta.content:
            print(chunk.choices[0].delta.content, end="", flush=True)
        if chunk.usage:
            print()
            print(chunk.usage.completion_tokens, "completion tokens")
```

The chunks arrive as the model produces them, unless your guardrail profile has output rails: the gateway then holds the whole answer so those rails can read it, and the chunks arrive in one burst. With `include_usage` set, the last chunk carries `usage` and an empty `choices` list, which is why the loop checks `chunk.choices` first.

## Verify

1. `models.list()` returns your tenant's models rather than OpenAI's catalogue; only the models the AI Gateway serves appear.
2. `completion.model` names the model that answered and `completion.usage` carries the token counts. The streamed reply's last chunk carries the same usage when `include_usage` is set.
3. The key's row on the Developer Hub's **API keys** tab shows a new last-used time, and each completion appears in the Control Room's **AI Gateway** under **Logs** as a line naming the provider, model, tier, tokens, cost and latency. That line does not name the key or the user.

## What does not carry over

- **Only the model routes.** The public API serves chat completions, completions, embeddings, rerank, image generation and the models list in the OpenAI shapes. The `responses`, `beta`, `files`, `batches`, `audio` and `moderations` clients on the same object have no route behind them.
- **`user` and `metadata`.** Both are accepted and discarded; neither reaches the model, the usage record or the logs. Attribute usage with the `X-Agent-ID` header instead.
- **`extra_body`.** Not a passthrough: ten sampling parameters for self-hosted engines are kept and every other key is dropped. [Use an OpenAI client library](/developers/guides/use-an-openai-client-library) lists them.
- **Streaming pace.** Chunks arrive as written, or in one burst after the output rails have read the answer.
- **Tools on a first turn.** With `tools` bound and `tool_choice` absent or `auto`, the first turn is upgraded to `required`, so the model must call a tool; `none` and a named tool are respected, and later turns are not upgraded. See [Call tools from a completion](/developers/guides/call-tools-from-a-completion).

## Troubleshooting

- **`AuthenticationError` on every call.** Check that the key is passed as `api_key` and that the base URL ends in `/public/v1`. The same error with a key that works elsewhere means the key lacks the route's scope: the gateway reports a missing scope as an invalid key. Chat completions need **Chat**; the models list needs **Read models**.
- **`RateLimitError`.** Read the error body. `RATE_LIMITED` is the key's own per-minute or per-day limit: wait for `X-RateLimit-Reset`. `rate_limit_exceeded` is a per-user, per-tenant or per-model limit inside the AI Gateway: the gateway's `Retry-After` is not relayed, so wait for the next clock minute. See [How limits combine](/developers/guides/how-limits-combine).
- **A call through `client.responses`, `client.files` or another non-model client fails.** Those clients have no route on the public API; use `chat.completions`, `completions`, `embeddings` or `images`.
- **`APITimeoutError` on a long answer.** The library's default timeout is ten minutes, and the generation routes have a longer budget than the sixty seconds that applies elsewhere. If you lowered `timeout` on the constructor, raise it for long answers.
- **`APIConnectionError`.** The host is wrong or a proxy blocks it. `curl https://<your tenant host>/public/v1/models` with the key in an `Authorization` header should return JSON.

## Related

- [Use an OpenAI client library](/developers/guides/use-an-openai-client-library)
- Chat and completions
- [OpenAI Node SDK](/developers/integrations/openai-node)
- [Integrations](/developers/integrations/overview)

Source: https://docs.katonic.ai/developers/integrations/openai-python
