# Embeddings and rerank

Embed text and rerank passages through the AI Gateway, with what input_type, dimensions and top_n do, which providers honour each, and where the responses differ from OpenAI's.

Two model routes serve retrieval work outside the knowledge engine. Embeddings turn text into vectors for an index you keep yourself; rerank scores a list of passages against a query. Both route by tier (`embedding` and `reranking`) or by model id, and both carry the fallback fields described in [Tiers and models](/developers/guides/tiers-and-models). If the passages already live in a Katonic knowledge source, [Search retrieval options](/developers/guides/search-options) reranks for you.

**Who can use it:** anyone holding an API key with the **Chat** or the **Completions** scope; either reaches both routes.

## The two routes

| | `POST /public/v1/embeddings` | `POST /public/v1/rerank` |
| --- | --- | --- |
| Input | `input`: one string or a list of strings | `query`, and `documents` as a list of strings |
| Model choice | `model`, or `tier` (default `embedding`) | `model`, or `tier` (default `reranking`) |
| Options | `encoding_format`, `dimensions`, `input_type` | `top_n` |
| Response | The OpenAI embedding list, `data[].embedding` and `usage`, plus `tier` and `provider` | `results[]` with `index` and `relevance_score`, plus `model`, `tier` and `provider` |
| What is recorded | Tokens and spend | Nothing: the route records no tokens and no spend |

## Quick start

Embed a list of passages for storage, then rerank a shortlist against a question:

**cURL**

```bash
curl -X POST "https://<your tenant host>/public/v1/embeddings" \
  -H "Authorization: Bearer kapi_..." \
  -H "Content-Type: application/json" \
  -d '{"input": ["Carer leave is ten days a year.", "Annual leave accrues monthly."], "input_type": "passage"}'

curl -X POST "https://<your tenant host>/public/v1/rerank" \
  -H "Authorization: Bearer kapi_..." \
  -H "Content-Type: application/json" \
  -d '{"query": "How much carer leave do I get?",
       "documents": ["Carer leave is ten days a year.", "Annual leave accrues monthly."], "top_n": 1}'
```

**Python**

```python

BASE = "https://<your tenant host>/public/v1"
HEADERS = {"Authorization": "Bearer kapi_..."}
passages = ["Carer leave is ten days a year.", "Annual leave accrues monthly."]

emb = requests.post(f"{BASE}/embeddings", headers=HEADERS,
                    json={"input": passages, "input_type": "passage"}).json()
vectors = [row["embedding"] for row in emb["data"]]          # same order as passages
print(len(vectors), "vectors of", len(vectors[0]), "dims from", emb["provider"])

rr = requests.post(f"{BASE}/rerank", headers=HEADERS, json={
    "query": "How much carer leave do I get?", "documents": passages, "top_n": 1}).json()
best = rr["results"][0]
print(passages[best["index"]], round(best["relevance_score"], 3), "scored by", rr["model"])
```

**TypeScript**

```typescript
const BASE = "https://<your tenant host>/public/v1";
const headers = { Authorization: "Bearer kapi_...", "Content-Type": "application/json" };
const passages = ["Carer leave is ten days a year.", "Annual leave accrues monthly."];

const emb = await (await fetch(`${BASE}/embeddings`, {
  method: "POST", headers, body: JSON.stringify({ input: passages, input_type: "passage" }),
})).json();
const vectors: number[][] = emb.data.map((row: { embedding: number[] }) => row.embedding);
console.log(vectors.length, "vectors of", vectors[0].length, "dims from", emb.provider);

const rr = await (await fetch(`${BASE}/rerank`, {
  method: "POST", headers,
  body: JSON.stringify({ query: "How much carer leave do I get?", documents: passages, top_n: 1 }),
})).json();
const best = rr.results[0];
console.log(passages[best.index], best.relevance_score.toFixed(3), "scored by", rr.model);
```

## The options

| Field | Route | What it does |
| --- | --- | --- |
| `input_type` | embeddings | `query` or `passage`, for models that embed questions and documents differently. Forwarded to the providers that distinguish them, each in its own vocabulary: NVIDIA (which assumes `query` when you send nothing), Cohere, Google and Voyage. Every other provider drops it. |
| `dimensions` | embeddings | Forwarded when set; the provider decides whether to honour it. |
| `encoding_format` | embeddings | Forwarded when set. |
| `user` | embeddings | Accepted and discarded. |
| `top_n` | rerank | Return only the best `n` results. Left out, every document comes back scored. |

Embedding rows come back in the order of `input`. A rerank result carries the `index` of the document in your list and a `relevance_score` between 0 and 1; use `index` to get back to the text. Both responses carry the `served_model` headers, so a fallback is visible.

## Limits and behaviour

- Pin `model` for anything you store. A tier fallback can answer with a different embedding model, and its vectors do not match the ones already in your index; `served_model` says which one answered.
- There is no per-request cap on the number of inputs or documents beyond the 64 MiB body limit. Provider limits still apply and come back as provider errors.
- Embedding tokens count toward the AI Gateway's tokens-per-minute limit and are priced into budgets. Rerank calls are checked against budgets before they run but add nothing to them, because the route returns no token count.
- The gateway's per-model rate limit is keyed on the tier name when no model is named.
- An empty `data` list from a provider is treated as a failure and the next candidate in the tier is tried.
- The checks run in the order budget, rate limit, token quota, so a call over budget is refused as `budget_exceeded` even when it is also over the rate limit. See [How limits combine](/developers/guides/how-limits-combine).

## Troubleshooting

- **422 with `no_models_for_tier`**: no model is tagged `embedding` or `reranking` for the tenant. An administrator tags one on the AI Gateway's Models page.
- **403 with `FORBIDDEN`**: the key has neither the Chat nor the Completions scope.
- **`dimensions` had no effect**: the provider that served the call does not support it; check `served_model` and pin one that does.
- **Two embeddings of the same text differ between calls**: a fallback served the second call. Read `X-Fallback-Reason` and pin the model.
- **A rerank `relevance_score` sits near 0.5 for everything**: the served model returned raw logits, which the gateway squashes into the 0 to 1 range; compare scores within one call, not across models.

## Related

- [Tiers and models](/developers/guides/tiers-and-models)
- [Search retrieval options](/developers/guides/search-options)
- Embeddings, rerank and images
- [How limits combine](/developers/guides/how-limits-combine)

Source: https://docs.katonic.ai/developers/guides/embeddings-and-rerank
