# Configure the AI Gateway and models

Add model providers and their keys, put models into tiers with a primary and fallbacks, set rate limits and watch usage and cost through the AI Gateway.

Every model call an agent makes goes through the AI Gateway. In this stage you connect the providers your organisation may use, decide which models sit in each routing tier, cap how fast anyone can call them, and learn where to watch the traffic.

**Who can use it:** the **Admin** role or the **Operator** grant, in a tenant licensed for the LLM gateway. On AI Cloud, providers and models are managed by your distributor.

Video: Configure the AI Gateway and Models (3:06), https://katonicdocsmedia.blob.core.windows.net/docs/configure-the-ai-gateway/c5abf362212d/index.html

## Prerequisites

- API keys for the providers you plan to use, or the URL of a self-hosted endpoint.
- To import the provider catalogue, Ops Hub connected under **Platform Settings**; see [Set up platform settings](/get-started/platform-settings).

## Add a provider

  
**Step 1. Open the gateway**

In the Control Room, open **AI Gateway**. A header chip says whether Ops Hub is connected, next to the buttons **Rate Limits**, **Logs** and **Settings**. The **Providers** tab opens first, with tiles counting providers, models and providers **With API Keys** and **Without Keys**, then request, cost, success-rate and latency tiles.

    ![The AI Gateway on its Providers tab: header chips for the Ops Hub connection and the provider and model counts, the Rate Limits, Logs and Settings buttons, the CONFIGURE tabs Providers, Models and Tiers and the OBSERVE tabs Usage and Costs, tiles for Providers, Total Models, With API Keys and Without Keys, then Total Requests, Total Cost, Avg Success Rate and Max Avg Latency, the Search providers box with Sync from Ops Hub and Add Provider, and provider rows each showing a KEY badge, a health badge, a model count, a success rate and a latency.](https://docs.katonic.ai/img/get-started/configure-the-ai-gateway/gateway-providers.png)
*The Providers tab of the AI Gateway. Each row shows whether the provider has a key and how healthy it is; Refresh Catalog imports the catalogue and Add Provider adds one by hand.*

  
**Step 2. Import or add providers**

**Refresh Catalog** imports the provider catalogue; when the list is empty and Ops Hub is reachable, the page does this once by itself. **Add Provider** opens a dialog: enter a **Provider Name**, pick a **Provider Type** and set the **Base URL**, which is prefilled for hosted services, built from fields such as a region or resource name for some, and required for a self-hosted endpoint. Add an **API Key** (optional here; **Test** checks it), use **Discover Models** to list the provider's models and tick the ones to import with a tier each, then select **Add Provider**.

  
**Step 3. Configure a key**

Rows without a credential show **NO KEY** and a **Configure Key** button, which opens **Manage API Key**. Enter the **API Key**, choose its **Environment**, select **Validate**, then **Save Key**: one key per provider, shared by all its models. A keyed row shows **KEY** and a health badge, **Healthy**, **Degraded** or **Unhealthy**, judged against success-rate thresholds set under **Settings**. **PLATFORM-MANAGED** marks a provider created by the platform's own LLM deployments; it needs no key. Select a row to open the provider's page.

## Enable models and assign tiers

The **Models** tab lists every model under three origins, **Provider**, **Deployed** and **Hosted**, with **Search models...** and an **All Tiers** filter. The columns are **Model**, **Tier**, **Priority**, **Context**, **Cost ($/1M)**, **Status**, **Health** and **Latency**, and **Status** reads **Active**, **Disabled** or **No Key**. The pencil at the end of a row opens **Edit**, where you set the model's **Tier**, switch it between **Enabled** and **Disabled** and record its costs. A provider's page adds models with **Add Model** or **Discover**.

A tier is an abstract routing target: when an agent asks for **Reasoning**, the gateway routes to the highest-priority healthy model in that tier. The **Tiers** tab lists **Reasoning**, **Fast**, **Balanced**, **Guard**, **Embedding**, **Reranking**, **Stt**, **Tts**, **Vision** and **Code**. In each, the top model is **PRIMARY** and the rest are fallbacks; use **Move up (higher priority)** and **Move down (lower priority)** or drag to reorder, and **Tier settings** (the gear) to pick the **Primary Model**. Models that are disabled, have no API key or whose key has not been validated are excluded from routing and counted as inactive.

![The Tiers tab of the AI Gateway: a sentence explaining that tiers are routing targets, then a section per tier, Reasoning first, with its model count and inactive count, and the models in routing order, the first marked PRIMARY and the rest FALLBACK, each with its provider, origin, lanes, status and health.](https://docs.katonic.ai/img/get-started/configure-the-ai-gateway/gateway-tiers.png)
*Each tier is a routing target. The top model in a tier is the primary and the others are fallbacks; Tier settings picks the primary, and the arrows or dragging change the order.*

## Set rate limits

Select **Rate Limits**. **Rate Limit Configuration** has three scopes, **Per User**, **Per Tenant** and **Per Model**, each with **Requests/Min (RPM)** and **Tokens/Min (TPM)**. The dialog describes them as sliding windows; the gateway counts them in fixed one-minute windows, so a burst that empties a minute's allowance is refused until the next minute begins. The shipped values are 30 and 60,000 per user, 200 and 500,000 per tenant, and 100 and 200,000 per model. **Save Changes** applies yours; **Reset Defaults** restores those values.

![The Rate Limit Configuration dialog: a line saying limits are counted in fixed one-minute windows, then the Per User and Per Tenant sections with their Requests/Min (RPM) and Tokens/Min (TPM) fields, and the top of the Per Model section.](https://docs.katonic.ai/img/get-started/configure-the-ai-gateway/gateway-rate-limits.png)
*Rate Limits sets requests and tokens per minute for each user, for the whole tenant and per model. Save Changes applies them; Reset Defaults restores the shipped values.*

## Watch it

**Usage** shows requests, success rate, latency and provider health for a time range, plus traffic by origin. **Costs** shows total and per-request cost, a breakdown by model and by origin, and budget alerts. **Logs** lists recent gateway requests, each opening to its **Request Info** and **Tokens & Cost**. Budgets belong to the next stage: see [Govern the platform day to day](/get-started/govern-day-to-day).

## Verify

On **Tiers**, the tier your agents use lists at least one model marked **PRIMARY** with the status **Active**, and its provider row reads **KEY** and **Healthy**. In the Work Room, a message to an agent gets an answer, and **Usage** counts the request.

## Troubleshooting

- **Ops Hub is not connected. Please connect Ops Hub from Platform Settings to sync providers.**: connect Ops Hub on its Platform Settings tab, or add providers by hand.
- **No providers configured yet.** or **No models configured. Add a provider first.**: select **Refresh Catalog** or **Add Provider**.
- **Invalid API key.** or **Saved key could not be decrypted. Re-enter the key.**: fix the key in **Manage API Key**.
- **Provider is unreachable. Check the Base URL and network.**, **Base URL is required for this provider.** or **Base URL is invalid. Check for unresolved placeholders or missing scheme.**: correct the endpoint on the provider's page.
- **Provider's TLS certificate is not trusted.**: for a trusted internal endpoint, tick the **TLS verification** option in the provider dialog to skip certificate checks.
- **Provider catalog is empty. Sync providers, then retry.**: run **Refresh Catalog**; if the catalogue was never pushed to your workspace, ask your platform operator.
- **Usage** or **Costs** shows a lock: your account lacks the permission or licence to view usage and cost data; the panel says which.

## Go deeper

The Control Room pages behind this journey, each with every field, state and message:

- AI Gateway
- Providers
- Models and tiers
- Usage and costs

## Related

- [Go live](/get-started/go-live)
- [Connect your organisation's knowledge](/get-started/connect-knowledge)
- [Govern the platform day to day](/get-started/govern-day-to-day)
- Admin Guide

Source: https://docs.katonic.ai/get-started/configure-the-ai-gateway
