Skip to main content

​
Migrating from LiteLLM

LiteLLM’s proxy and KnoxCall’s AI Gateway sit in the same place in your stack and solve overlapping problems, so most of a config.yaml maps across mechanically. There is a command for that.
That is a dry run. It reads the file, prints exactly what it would create, and writes nothing. --apply is the deliberate second step.

​
What maps onto what

​
Load balancing collapses into one agent

This is the mapping worth understanding, because it changes your topology for the better. In LiteLLM, listing the same model_name twice is the load balancer:
KnoxCall expresses that as one agent with two weighted routes, so it needs one capability token rather than two, and one budget rather than two that have to be kept in step. See Routing, Retries & Fail-over.

​
Your model names survive

Existing callers ask for "model": "fast". Rather than make you edit every call site on day one, the importer writes a gateway-level model alias mapping fast → the real upstream id. Aliases resolve before the per-agent model policy, so an alias target still has to pass that agent’s allow/deny list.

​
What it will not do: your keys

The importer never reads an API key out of your config, and never transmits one. It records how each entry referenced its credential and leaves you to create the KnoxCall secret:
If an entry carries a literal key rather than an os.environ/ reference, the importer says so and does not read it — and tells you to treat that key as exposed, because it is sitting in a file. Rotate it at the provider when you create the KnoxCall secret. That is not squeamishness. A migration tool that quietly ingests a plaintext key from a checked-in config has copied that key into a second system, which is an odd thing for a credential-custody product to do to you on your first day.

​
What it will not do: guess

Two refusals you may hit, both deliberate: An unprefixed model. LiteLLM treats model: gpt-4o with no provider prefix as OpenAI. The importer will not inherit that default for you — pointing an agent at the wrong provider with the wrong key is a worse outcome than a warning. Prefix it (openai/gpt-4o) or set the provider after import. A budget duration we do not have. KnoxCall has a daily cap and a monthly cap. budget_duration: 7d is neither, so the budget is not imported and the report says so. Dividing by seven to fake a daily rate would put a number in front of you that you never chose.

​
The YAML it can read

The importer parses a YAML subset — nested maps, lists of maps, scalars, simple inline lists — which covers the LiteLLM configs people actually have. It has no third-party YAML dependency, because knoxcall ships with zero runtime dependencies and a credential-handling CLI is the wrong place to grow a transitive tree. Anything outside that subset — anchors, aliases, merge keys, block scalars, multi-document files — is a hard error naming the line:
It refuses rather than skips, on purpose. An importer that silently dropped a model_list entry would hand you a plan that looks complete and is not; you would migrate, believe you were done, and find the missing model in production. Being told about line 14 costs a minute.

​
Applying it

Creates the gateway, the agents and the model aliases. Exit codes: 0 success, 1 something failed while applying (partial creation is possible, and every failure is named so you can retry just those), 2 the input was unusable and nothing was written. Then, for each agent: create a secret holding the provider key and attach it. At that point you have what LiteLLM gave you, plus per-agent capability tokens, mid-stream PII redaction, a prompt firewall, and per-call attribution.

​
What you gain that has no LiteLLM equivalent

Worth knowing before you plan the cutover, because these are the reasons to do it rather than the mechanics:
  • Your app never holds a provider key. It holds a KnoxCall capability token scoped to a provider, a model set, a spend cap and a time window. The real key is injected at the egress hop. See Capability keys.
  • Mid-stream PII redaction — entities are removed inside the SSE stream, including ones split across two frames, without buffering the response.
  • A prompt firewall that can refuse a request, and your own scanner if you already have one.
  • Per-call attribution and audit — which token, which user, which team, what it cost.
One thing to check before you cut over: LiteLLM supports more providers than KnoxCall’s fourteen. Run the dry run first and read the warnings — anything that could not be mapped is named there, and an entry with an api_base will usually map as openai-compatible even when its prefix is unknown to us.