OpenAI Node SDK
Public API
The openai package for Node.js takes the same two settings as its Python sibling, a base URL and a key, and from then on runs every chat completion through your tenant's AI Gateway. It suits server-side code: route handlers, workers and scripts that hold the key on behalf of the application.
Who can use it: anyone holding an API key with the Chat scope; add Read models if the application lists models itself. The tenant must be licensed for the public API.
Prerequisites
- Node.js 18 or later and
npm install openai. Verified withopenai7.15.0. - Your tenant's public API address,
https://<your tenant host>/public/v1, from the top of Studio's Developer Hub. - A key from the Developer Hub's API keys tab. The Chat only preset is enough to complete; add Read models to list models.
Connect
import OpenAI from "openai";
const client = new OpenAI({
baseURL: "https://<your tenant host>/public/v1",
apiKey: process.env.KATONIC_API_KEY,
});
const models = await client.models.list();
console.log(models.data.slice(0, 5).map((m) => m.id));
const completion = await client.chat.completions.create({
model: "<a model id from the list>",
messages: [{ role: "user", content: "Summarise our onboarding guide in three bullet points." }],
max_tokens: 300,
});
console.log(completion.model, completion.usage?.completion_tokens);
console.log(completion.choices[0].message.content);
The file must be a module ("type": "module" in package.json, or a .mjs extension) for the top-level await. The library also reads OPENAI_API_KEY and OPENAI_BASE_URL from the environment when the options are omitted, so unchanged code runs against your tenant from a shell that exports both. To record the usage against one of your agents in analytics, pass defaultHeaders: { "X-Agent-ID": "<agent id>" } to the constructor.
Stream a reply
const stream = await client.chat.completions.create({
model: "<a model id from the list>",
messages: [{ role: "user", content: "Write a short welcome note for new starters." }],
max_tokens: 300,
stream: true,
stream_options: { include_usage: true },
});
for await (const chunk of stream) {
process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
if (chunk.usage) {
console.log();
console.log(chunk.usage.completion_tokens, "completion tokens");
}
}
The chunks arrive as the model produces them, unless your guardrail profile has output rails: the gateway then holds the whole answer so those rails can read it, and the chunks arrive in one burst. With include_usage set, the last chunk carries usage and an empty choices array, which is why the loop uses optional chaining.
Verify
models.list()returns your tenant's models rather than OpenAI's catalogue; only the models the AI Gateway serves appear.completion.modelnames the model that answered andcompletion.usagecarries the token counts. The streamed reply's last chunk carries the same usage wheninclude_usageis set.- The key's row on the Developer Hub's API keys tab shows a new last-used time, and each completion appears in the Control Room's AI Gateway under Logs as a line naming the provider, model, tier, tokens, cost and latency. That line does not name the key or the user.
What does not carry over
- Only the model routes. The public API serves chat completions, completions, embeddings, rerank, image generation and the models list in the OpenAI shapes. The
responses,beta,files,batches,audioandmoderationsclients on the same object have no route behind them. userandmetadata. Both are accepted and discarded; attribute usage with theX-Agent-IDheader instead.extra_body. Not a passthrough: ten sampling parameters for self-hosted engines are kept and every other key is dropped. Use an OpenAI client library lists them.- Streaming pace. Chunks arrive as written, or in one burst after the output rails have read the answer.
- Tools on a first turn. With
toolsbound andtool_choiceabsent orauto, the first turn is upgraded torequired, so the model must call a tool;noneand a named tool are respected, and later turns are not upgraded. See Call tools from a completion. - The browser. The library refuses to run in a browser unless you opt in, and opting in puts the key in the page. Keep it server-side and let your own backend hold the key.
Troubleshooting
AuthenticationErroron every call. Check that the key is passed asapiKeyand that the base URL ends in/public/v1. The same error with a key that works elsewhere means the key lacks the route's scope: the gateway reports a missing scope as an invalid key. Chat completions need Chat; the models list needs Read models.RateLimitError. Read the error body.RATE_LIMITEDis the key's own per-minute or per-day limit: wait forX-RateLimit-Reset.rate_limit_exceededis a per-user, per-tenant or per-model limit inside the AI Gateway: the gateway'sRetry-Afteris not relayed, so wait for the next clock minute. See How limits combine.SyntaxError: Cannot use import statement outside a module. Add"type": "module"topackage.jsonor rename the file to.mjs.- A call through
client.responses,client.filesor another non-model client fails. Those clients have no route on the public API; usechat.completions,completions,embeddingsorimages. APIConnectionError. The host is wrong or a proxy blocks it.curl https://<your tenant host>/public/v1/modelswith the key in anAuthorizationheader should return JSON.