> ## Documentation Index
> Fetch the complete documentation index at: https://docs.knoxcall.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Routing, Retries & Fail-over

> Retry on 429 with Retry-After, exponential backoff with jitter, weighted load-balancing across providers, and gateway-wide model aliases — configured per agent, enforced in the data plane.

# Routing, Retries & Fail-over

Your application points at one agent URL. What happens behind it — how many
providers, which one served this call, what happened when the first one said
"slow down" — is the agent's configuration, not your code.

## The default

Every agent starts with `routing_policy: {}`, and that means:

* **one attempt** per route, no retries;
* **fail over on 5xx** to `fallback_route_ids`, in order;
* everything else — 429, 4xx, 2xx — returned to you unchanged.

That is deliberate, and it is what an agent does until you ask for something
else. In particular a **429 comes straight back to you**: rate limits are
information your client may want to act on, and silently absorbing them into a
second provider's bill is not a decision the gateway makes for you.

## Retrying a rate limit

The one you almost certainly want:

```json theme={"dark"}
{
  "routing_policy": {
    "retry_on": ["429", "5xx"],
    "max_attempts": 3,
    "backoff_ms": 250,
    "backoff_multiplier": 2,
    "max_backoff_ms": 5000
  }
}
```

Now a 429 is retried against the same route — honouring the provider's own
`Retry-After` header — and if the route is still rate-limited after
`max_attempts`, the request moves to the next candidate.

`Retry-After` is honoured in **one direction only**: it can extend a wait, never
shorten one. A provider saying "retry after 1s" while the gateway is already
backing off 3.2s does not get hammered at 1s. Both
[RFC 9110 forms](https://www.rfc-editor.org/rfc/rfc9110#field.retry-after) are
understood — delta-seconds and an HTTP-date.

<Note>
  **Jitter is on by default.** Each delay is spread across 50–100% of its computed
  value, so a fleet of workers that all hit the same rate limit at the same moment
  does not retry in lockstep and re-trip it. Set `"jitter": false` if you need a
  deterministic schedule for a test.
</Note>

### Timeouts

`"retry_on": ["timeout"]` covers a transport failure — connection reset, DNS
failure, the upstream hanging up. It is separate from `5xx` because they call
for different appetites: a 502 usually means "this provider is having a bad
minute", a connection reset often means "that one packet was unlucky".

## Weighted load-balancing

Two keys on the same provider, or two providers entirely, split by weight:

```json theme={"dark"}
{
  "routing_policy": {
    "route_weights": {
      "<primary-route-id>": 3,
      "<fallback-route-id>": 1
    }
  }
}
```

75% of requests are served by the first route, 25% by the second — and the one
not chosen is still in the fail-over order behind it, so a weight spreads load
without giving up redundancy.

Three rules worth knowing:

* **A route you do not name keeps weight 1.** Adding a weight for one route does
  not silently drain the others.
* **Weight 0 drains a route completely** — no ordinary traffic and no fail-over
  traffic. That is what "park this provider" means; if you wanted "try it last",
  give it weight 1 against a much larger one.
* **Your token's scope still wins.** The capability scope on the phantom token is
  checked against *every* candidate before it is dialled, so a weight can never
  reach a route the token was not granted, and neither can a fail-over. Same for
  Live/Test: a `kc_test_` token resolves only sandbox routes at every hop.

## Model aliases

Set once on the gateway; every agent under it honours them:

```bash theme={"dark"}
curl -X PATCH https://api.knoxcall.com/v1/ai-gateway/gateways/$GATEWAY_ID \
  -H "Authorization: Bearer $KNOXCALL_TOKEN" \
  -H 'Content-Type: application/json' \
  -d '{"model_aliases": {"fast": "claude-haiku-4-5", "flagship": "claude-opus-5"}}'
```

Your services then ask for `"model": "fast"` and you change what that means in
one place. Chains work (`cheap` → `fast` → a real id); a cycle resolves back to
the model you asked for rather than looping.

Aliases resolve **before** the agent's model policy, which is the important part:
an alias target still has to pass that agent's allowlist and denylist. If the
resolution ran the other way round, `{"safe-model": "the-one-you-banned"}` would
be a one-line way around your own denylist.

## What it recorded

Every usage row now carries what actually happened:

| Column            | Meaning                                                                                                            |
| ----------------- | ------------------------------------------------------------------------------------------------------------------ |
| `served_route_id` | The route that produced the response — not necessarily the agent's primary, once weights or fail-over are in play. |
| `attempts`        | Total upstream dispatches for the request, including the first. `1` is the ordinary case.                          |

Both are `NULL` for a cache hit, because nothing was dispatched. `attempts > 1`
is the signal worth alerting on: it means a provider is making you pay for the
same answer twice.

## The limits, and why they exist

Every field is clamped server-side:

| Field                       | Ceiling |
| --------------------------- | ------- |
| `max_attempts`              | 5       |
| any single delay            | 30s     |
| a honoured `Retry-After`    | 60s     |
| total sleep for one request | 60s     |
| any `route_weights` value   | 1000    |

These are not tuning suggestions. A request waiting to retry is holding an API
worker, so an agent configured with `max_attempts: 10000` and
`backoff_ms: 600000` would be a self-inflicted outage — and a hostile upstream
answering `429` with `Retry-After: 86400` would be someone else's. The ceilings
are what make "configure your own retries" safe to offer at all.

`max_attempts` is also forced back to 1 whenever `retry_on` is empty, so a stored
policy never claims a retry that cannot happen. An unrecognised key inside
`routing_policy` is rejected with a 400 naming the field rather than stored and
ignored — a `retryOn` typo that reads as configured and does nothing is worse
than an error.