# LangChain

Use the ChatOpenAI class from langchain-openai against your tenant's public API so chains, agents and streams run through your tenant's AI Gateway with one base URL and one key.

LangChain's `ChatOpenAI` class speaks the chat completions protocol, and your tenant's public API answers it. Construct the model with a base URL and a key, and every chain, agent or graph that takes a chat model runs through your tenant's AI Gateway, with its tiers, guardrails, budgets and logs, and without a provider key of its own.

**Who can use it:** anyone holding an API key with the **Chat** scope. The tenant must be licensed for the public API.

## Prerequisites

- Python 3.10 or later and `pip install langchain-openai`. Verified with `langchain-openai` 1.6.2 on `langchain-core` 1.6.3.
- Your tenant's public API address, `https://<your tenant host>/public/v1`, from the top of Studio's **Developer Hub**.
- A key from the Developer Hub's **API keys** tab, and a model id your tenant serves. The models route lists them; [Use an OpenAI client library](/developers/guides/use-an-openai-client-library) shows the call.

## Connect

```python
from langchain_openai import ChatOpenAI

llm = ChatOpenAI(
    model="<a model id your tenant serves>",
    base_url="https://<your tenant host>/public/v1",
    api_key="kapi_YOUR_KEY",
    max_tokens=300,
)

reply = llm.invoke("Summarise our onboarding guide in three bullet points.")
print(reply.content)
print(reply.response_metadata["model_name"], reply.usage_metadata["output_tokens"])
```

`invoke` accepts a string, a list of messages or a prompt value and returns an `AIMessage`. Always pass `model`: the class's default is one of OpenAI's model names, which your tenant may not serve. The object drops into any place LangChain expects a chat model, for example a prompt piped into it:

```python
from langchain_core.prompts import ChatPromptTemplate

prompt = ChatPromptTemplate.from_messages([
    ("system", "You answer in one sentence."),
    ("human", "{question}"),
])
chain = prompt | llm
print(chain.invoke({"question": "What does the onboarding guide cover?"}).content)
```

To record the usage against one of your agents in analytics, add `default_headers={"X-Agent-ID": "<agent id>"}` to the constructor.

## Stream a reply

```python
for chunk in llm.stream("Write a short welcome note for new starters."):
    print(chunk.content, end="", flush=True)
```

The chunks arrive in one burst, not as the model produces them: the gateway holds the whole answer so the output guardrails can read it, then replays the chunks. Stream for the event shape, not for a typing effect. Some chunks carry no content, the first with only a role and the last with only metadata, so a consumer that concatenates `chunk.content` gets the whole answer without special cases.

## Verify

1. `reply.response_metadata` names the model that answered and `reply.usage_metadata` carries the token counts, as the gateway returned them.
2. The key's row on the Developer Hub's **API keys** tab shows a new last-used time.
3. Each call appears in the Control Room's **AI Gateway** under **Logs** as a line naming the provider, model, tier, tokens, cost and latency. That line does not name the key or the user.

## What does not carry over

- **Tools on a first turn.** After `bind_tools`, a first turn whose tool choice is absent or `auto` is upgraded to `required`, so the model must call a tool; `none` and a named tool are respected, and later turns are not upgraded. An agent loop built on the model behaves accordingly. See [Call tools from a completion](/developers/guides/call-tools-from-a-completion).
- **`user` and `metadata`.** Both are accepted and discarded; attribute usage with the `X-Agent-ID` header instead.
- **Streaming pace.** Chunks arrive in one burst after the guardrails have read the answer.
- **Only the model routes.** Chat completions, completions, embeddings, rerank, image generation and the models list are served in the OpenAI shapes; nothing else on OpenAI's API is. [Embeddings and rerank](/developers/guides/embeddings-and-rerank) covers the embedding models your tenant serves.

## Troubleshooting

- **`AuthenticationError` on every call.** Check the key and that the base URL ends in `/public/v1`. The same error with a key that works elsewhere means the key lacks the **Chat** scope: the gateway reports a missing scope as an invalid key.
- **`RateLimitError`.** `RATE_LIMITED` in the body is the key's own per-minute or per-day limit: wait for `X-RateLimit-Reset`. `rate_limit_exceeded` is a per-user, per-tenant or per-model limit inside the AI Gateway: the gateway's `Retry-After` is not relayed, so wait for the next clock minute. See [How limits combine](/developers/guides/how-limits-combine).
- **An error names a model you did not choose.** `model` was omitted and the class used its own default. Pass a model id your tenant serves.
- **Errors surface late while debugging.** The class retries transient failures twice before raising. Construct it with `max_retries=0` while you diagnose a connection, then restore the default.
- **A chain works in one environment and fails with a connection error in another.** The host is wrong or a proxy blocks it. `curl https://<your tenant host>/public/v1/models` with the key in an `Authorization` header should return JSON.

## Related

- [Use an OpenAI client library](/developers/guides/use-an-openai-client-library)
- Chat and completions
- [LlamaIndex](/developers/integrations/llamaindex)
- [Integrations](/developers/integrations/overview)

Source: https://docs.katonic.ai/developers/integrations/langchain
