Skip to main content

OpenAI Node SDK

Public API

The openai package for Node.js takes the same two settings as its Python sibling, a base URL and a key, and from then on runs every chat completion through your tenant's AI Gateway. It suits server-side code: route handlers, workers and scripts that hold the key on behalf of the application.

Who can use it: anyone holding an API key with the Chat scope; add Read models if the application lists models itself. The tenant must be licensed for the public API.

Prerequisites​

  • Node.js 18 or later and npm install openai. Verified with openai 7.15.0.
  • Your tenant's public API address, https://<your tenant host>/public/v1, from the top of Studio's Developer Hub.
  • A key from the Developer Hub's API keys tab. The Chat only preset is enough to complete; add Read models to list models.

Connect​

import OpenAI from "openai";

const client = new OpenAI({
baseURL: "https://<your tenant host>/public/v1",
apiKey: process.env.KATONIC_API_KEY,
});

const models = await client.models.list();
console.log(models.data.slice(0, 5).map((m) => m.id));

const completion = await client.chat.completions.create({
model: "<a model id from the list>",
messages: [{ role: "user", content: "Summarise our onboarding guide in three bullet points." }],
max_tokens: 300,
});
console.log(completion.model, completion.usage?.completion_tokens);
console.log(completion.choices[0].message.content);

The file must be a module ("type": "module" in package.json, or a .mjs extension) for the top-level await. The library also reads OPENAI_API_KEY and OPENAI_BASE_URL from the environment when the options are omitted, so unchanged code runs against your tenant from a shell that exports both. To record the usage against one of your agents in analytics, pass defaultHeaders: { "X-Agent-ID": "<agent id>" } to the constructor.

Stream a reply​

const stream = await client.chat.completions.create({
model: "<a model id from the list>",
messages: [{ role: "user", content: "Write a short welcome note for new starters." }],
max_tokens: 300,
stream: true,
stream_options: { include_usage: true },
});

for await (const chunk of stream) {
process.stdout.write(chunk.choices[0]?.delta?.content ?? "");
if (chunk.usage) {
console.log();
console.log(chunk.usage.completion_tokens, "completion tokens");
}
}

The chunks arrive as the model produces them, unless your guardrail profile has output rails: the gateway then holds the whole answer so those rails can read it, and the chunks arrive in one burst. With include_usage set, the last chunk carries usage and an empty choices array, which is why the loop uses optional chaining.

Verify​

  1. models.list() returns your tenant's models rather than OpenAI's catalogue; only the models the AI Gateway serves appear.
  2. completion.model names the model that answered and completion.usage carries the token counts. The streamed reply's last chunk carries the same usage when include_usage is set.
  3. The key's row on the Developer Hub's API keys tab shows a new last-used time, and each completion appears in the Control Room's AI Gateway under Logs as a line naming the provider, model, tier, tokens, cost and latency. That line does not name the key or the user.

What does not carry over​

  • Only the model routes. The public API serves chat completions, completions, embeddings, rerank, image generation and the models list in the OpenAI shapes. The responses, beta, files, batches, audio and moderations clients on the same object have no route behind them.
  • user and metadata. Both are accepted and discarded; attribute usage with the X-Agent-ID header instead.
  • extra_body. Not a passthrough: ten sampling parameters for self-hosted engines are kept and every other key is dropped. Use an OpenAI client library lists them.
  • Streaming pace. Chunks arrive as written, or in one burst after the output rails have read the answer.
  • Tools on a first turn. With tools bound and tool_choice absent or auto, the first turn is upgraded to required, so the model must call a tool; none and a named tool are respected, and later turns are not upgraded. See Call tools from a completion.
  • The browser. The library refuses to run in a browser unless you opt in, and opting in puts the key in the page. Keep it server-side and let your own backend hold the key.

Troubleshooting​

  • AuthenticationError on every call. Check that the key is passed as apiKey and that the base URL ends in /public/v1. The same error with a key that works elsewhere means the key lacks the route's scope: the gateway reports a missing scope as an invalid key. Chat completions need Chat; the models list needs Read models.
  • RateLimitError. Read the error body. RATE_LIMITED is the key's own per-minute or per-day limit: wait for X-RateLimit-Reset. rate_limit_exceeded is a per-user, per-tenant or per-model limit inside the AI Gateway: the gateway's Retry-After is not relayed, so wait for the next clock minute. See How limits combine.
  • SyntaxError: Cannot use import statement outside a module. Add "type": "module" to package.json or rename the file to .mjs.
  • A call through client.responses, client.files or another non-model client fails. Those clients have no route on the public API; use chat.completions, completions, embeddings or images.
  • APIConnectionError. The host is wrong or a proxy blocks it. curl https://<your tenant host>/public/v1/models with the key in an Authorization header should return JSON.

Describes the platform as of .

Was this page helpful?