Skip to main content

Vercel AI SDK

Public API

The Vercel AI SDK talks to any OpenAI-compatible server through its @ai-sdk/openai-compatible provider. Create the provider with your tenant's base URL and the key in a header, take a chat model from it, and generateText and streamText run through your tenant's AI Gateway from a route handler, a worker or a script.

Who can use it: anyone holding an API key with the Chat scope. The tenant must be licensed for the public API.

Prerequisites​

  • Node.js 18 or later and npm install ai @ai-sdk/openai-compatible. Verified with ai 7.0.99 and @ai-sdk/openai-compatible 3.0.48.
  • Your tenant's public API address, https://<your tenant host>/public/v1, from the top of Studio's Developer Hub.
  • A key from the Developer Hub's API keys tab, and a model id your tenant serves. The models route lists them; Use an OpenAI client library shows the call.

Connect​

import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
import { generateText } from "ai";

const katonic = createOpenAICompatible({
name: "katonic",
baseURL: "https://<your tenant host>/public/v1",
headers: { Authorization: `Bearer ${process.env.KATONIC_API_KEY}` },
});
const model = katonic.chatModel("<a model id your tenant serves>");

const result = await generateText({
model,
prompt: "Summarise our onboarding guide in three bullet points.",
maxOutputTokens: 300,
});
console.log(result.text);
console.log(result.usage.outputTokens, "completion tokens");

Use this provider rather than @ai-sdk/openai: that package's default model class calls OpenAI's Responses route, which the public API does not serve. name labels the provider in the SDK's metadata, and the model id is passed through to your tenant unchanged. The file must be a module ("type": "module" in package.json, or a .mjs extension) for the top-level await.

Stream a reply​

import { streamText } from "ai";

const stream = streamText({
model,
prompt: "Write a short welcome note for new starters.",
maxOutputTokens: 300,
});
for await (const part of stream.textStream) {
process.stdout.write(part);
}

In a route handler, hand the result to the SDK's response helpers instead of iterating it yourself. The parts arrive in one burst, not as the model produces them: the gateway holds the whole answer so the output guardrails can read it, then replays the chunks. Stream for the event shape, not for a typing effect.

Verify​

  1. result.text is the model's answer and result.usage.outputTokens carries the completion tokens the gateway counted.
  2. The key's row on the Developer Hub's API keys tab shows a new last-used time.
  3. Each call appears in the Control Room's AI Gateway under Logs as a line naming the provider, model, tier, tokens, cost and latency. That line does not name the key or the user.

What does not carry over​

  • The OpenAI provider's defaults. @ai-sdk/openai assumes OpenAI's Responses route; the public API serves chat completions, completions, embeddings, rerank, image generation and the models list in the OpenAI shapes, so stay on the OpenAI-compatible provider.
  • Tools on a first turn. With tools passed to generateText or streamText and no tool choice set, the first turn is upgraded to required, so the model must call a tool; later turns are not upgraded. See Call tools from a completion.
  • Streaming pace. Parts arrive in one burst after the guardrails have read the answer.
  • The browser. The provider runs anywhere fetch does, but the key would then live in the page. Keep it in a route handler or server component and let your own backend hold the key.

Troubleshooting​

  • APICallError with status 401. Check the header and that the base URL ends in /public/v1. The same error with a key that works elsewhere means the key lacks the Chat scope: the gateway reports a missing scope as an invalid key.
  • APICallError with status 429. RATE_LIMITED in the body is the key's own per-minute or per-day limit: wait for X-RateLimit-Reset. rate_limit_exceeded is a per-user, per-tenant or per-model limit inside the AI Gateway: the gateway's Retry-After is not relayed, so wait for the next clock minute. See How limits combine.
  • Invalid JSON response. The address returned something other than JSON, usually a sign-in or error page from the wrong host or path. The base URL must end in /public/v1.
  • A 404 from @ai-sdk/openai. Its default model class calls the Responses route. Switch to @ai-sdk/openai-compatible as above.
  • SyntaxError: Cannot use import statement outside a module. Add "type": "module" to package.json or rename the file to .mjs.

Describes the platform as of .

Was this page helpful?