Call tools from a completion
The chat completions route accepts OpenAI-shaped tools: you describe functions, the model answers with tool calls instead of text, you run them and send the results back for the final answer. The gateway forwards the shapes but is not a pass-through. It decides when the model must call a tool, strips tools a provider rejects, repairs malformed calls from some open models, and says what it changed in response headers.
Who can use it: anyone holding an API key with the Chat scope. These are your own functions, run by your code; the platform's governed tool catalogue is a separate route, covered in Execute a governed tool.
What the gateway changes
| You send | What happens |
|---|---|
tools, up to 128 | Forwarded. If the provider answers that the model cannot use tools, the call is retried once without tools, tool_choice and their relatives. The gateway marks such a response with X-Tools-Stripped, a header the public edge does not relay on the current build, so through the public API the only sign is an answer without tool_calls. |
tool_choice absent or auto | On the first turn, with real tools bound, upgraded to required: the model must call a tool. Relaxed back to auto once the conversation holds a tool result or an assistant turn in the current exchange, and for greetings. |
tool_choice: "required" | Kept, with the same relaxations. |
tool_choice: "none" or a named function | Respected as sent. |
parallel_tool_calls | Dropped; the gateway does not know the field. |
A tool message with tool_call_id | Forwarded. Several system messages are merged into one leading system message for every provider. |
Self-hosted engines that use guided decoding never receive a forced choice; they get auto. A model that refuses a forced choice gets auto with X-Tool-Choice-Relaxed: model_rejects_forced_choice, and a provider that rejects the value gets auto on a retry, disclosed as X-Tool-Choice-Relaxed: provider_rejected.
Quick start
Define one function, let the model call it, run it, and send the result back:
- cURL
- Python
- TypeScript
curl -X POST "https://<your tenant host>/public/v1/chat/completions" \
-H "Authorization: Bearer kapi_..." \
-H "Content-Type: application/json" \
-d '{"tier": "balanced",
"messages": [{"role": "user", "content": "Is order 4471 shipped?"}],
"tools": [{"type": "function", "function": {"name": "order_status",
"description": "Look up an order by id",
"parameters": {"type": "object", "properties": {"order_id": {"type": "string"}}, "required": ["order_id"]}}}]}'
The reply carries choices[0].message.tool_calls; run the function and send the whole exchange back with a tool message carrying the same tool_call_id.
import json, requests
BASE = "https://<your tenant host>/public/v1"
HEADERS = {"Authorization": "Bearer kapi_..."}
TOOLS = [{"type": "function", "function": {"name": "order_status", "description": "Look up an order by id",
"parameters": {"type": "object", "properties": {"order_id": {"type": "string"}}, "required": ["order_id"]}}}]
def order_status(order_id: str) -> dict:
return {"order_id": order_id, "status": "shipped", "carrier": "DHL"}
messages = [{"role": "user", "content": "Is order 4471 shipped?"}]
first = requests.post(f"{BASE}/chat/completions", headers=HEADERS,
json={"tier": "balanced", "messages": messages, "tools": TOOLS}).json()
reply = first["choices"][0]["message"]
messages.append(reply)
for call in reply.get("tool_calls") or []:
args = json.loads(call["function"]["arguments"])
messages.append({"role": "tool", "tool_call_id": call["id"], "content": json.dumps(order_status(**args))})
final = requests.post(f"{BASE}/chat/completions", headers=HEADERS,
json={"tier": "balanced", "messages": messages, "tools": TOOLS}).json()
print(final["choices"][0]["message"]["content"])
const BASE = "https://<your tenant host>/public/v1";
const headers = { Authorization: "Bearer kapi_...", "Content-Type": "application/json" };
const tools = [{ type: "function", function: { name: "order_status", description: "Look up an order by id",
parameters: { type: "object", properties: { order_id: { type: "string" } }, required: ["order_id"] } } }];
const orderStatus = (order_id: string) => ({ order_id, status: "shipped", carrier: "DHL" });
const complete = (messages: unknown[]) =>
fetch(`${BASE}/chat/completions`, { method: "POST", headers, body: JSON.stringify({ tier: "balanced", messages, tools }) }).then((r) => r.json());
const messages: any[] = [{ role: "user", content: "Is order 4471 shipped?" }];
const reply = (await complete(messages)).choices[0].message;
messages.push(reply);
for (const call of reply.tool_calls ?? []) {
const args = JSON.parse(call.function.arguments);
messages.push({ role: "tool", tool_call_id: call.id, content: JSON.stringify(orderStatus(args.order_id)) });
}
console.log((await complete(messages)).choices[0].message.content);
Read the response
A turn that calls tools has finish_reason set to tool_calls and choices[0].message.tool_calls, each with an id, type of function, and function.name and function.arguments, the arguments as a JSON string to parse. On a stream the tool-call deltas pass through with their index and function filled in; the whole answer is buffered and replayed anyway, as every stream on this route is. The characters of the arguments count toward completion_tokens.
For several open-weight model families the gateway repairs a call the model wrote as text into a structured call. When a streamed call cannot be repaired, the gateway drops it and appends a notice to the content instead of failing the turn, so parse tool_calls, not the text.
Limits and behaviour
- With tools bound, a first-turn question is answered with a tool call, not text, unless you send
tool_choiceasnoneor name a function. Sendnonefor turns where a text answer is wanted. - A model answer with no content, no tool call and no refusal counts as a failure and moves the call to the next candidate in the tier.
- The gateway never runs a function. Whatever
tool_callsnames, your code executes it and decides what to send back. - The model may call a tool that is not in your list only if the provider does; the gateway does not filter the names on the way back.
- The output guardrails read the text of the final answer, not the tool arguments; the tool rails of a guardrail profile apply to the platform's governed tools, not to these.
Troubleshooting
- The model always calls a tool on the first turn, even for small talk: the upgrade to
required. Sendtool_choiceasnonefor those turns. tool_callsis missing although you sent tools: the served model cannot use tools and the call was retried without them; theX-Tools-Strippedheader that says so does not reach public API callers on the current build. Pin a model whosecapabilities.toolsis true on the models route.- A forced
tool_choicecame back as text: the model or the provider refused the forced choice and was givenauto. The gateway records this in anX-Tool-Choice-Relaxedheader that is not relayed on the current build, so handle both shapes. function.argumentsis not valid JSON: the model wrote it badly and the repair did not apply. Validate before running anything and send the error back as the tool result so the model can retry.- Several system prompts behave as one: they are merged before the provider sees them; put per-turn instructions in a user message.
The same job elsewhere