Streaming Guarantees
Most gateways can only inspect a streamed AI response after it finishes, or log what they saw. KnoxCall redacts inside the live SSE stream. This page states the per-mode guarantee and publishes the latency cost — a number competitors who buffer or don’t enforce mid-stream can’t put on the table.The three modes
Each agent picks apii_streaming_mode (default holdback).
All three record detections; only
holdback and buffer alter the bytes the client sees.
Why holdback is the hard one (and the differentiator)
A sensitive token can arrive split across SSE frames —"...SSN 123-" in one delta, "45-6789..." in the next. A naïve mid-stream redactor emits the first half before it can tell it’s PII. KnoxCall’s hold-back FSM keeps the last N characters buffered until enough context has arrived to decide, then splices any redaction back into the SSE frame and re-serialises it — so the client never sees a partial entity, and framing is never corrupted. Bedrock punts (stream:false required); Cloudflare buffers; Portkey’s streaming guardrails are log-only. KnoxCall enforces.
Published benchmark
scripts/bench-ai-gateway-streaming.ts drives a representative Anthropic response (120 text deltas seeded with email/SSN/credit-card entities) through each mode and a raw passthrough baseline, and records p50/p95 TTFB (time-to-first-emitted-byte) and total drain time. The snapshot lives at scripts/bench-ai-gateway-streaming.results.json and CI diffs it, so a regression in the flagship path fails the build.
This is a CPU-bound micro-benchmark of the redaction Transform itself (no network) — it isolates the overhead the gateway adds on top of the hundreds of milliseconds an LLM already takes. Representative numbers on commodity CI hardware:
The headline: mid-stream redaction (holdback) adds sub-millisecond first-byte latency — noise against LLM response times measured in hundreds of milliseconds. You get real, enforced, mid-stream PII protection essentially for free on the latency budget.
Absolute numbers are hardware-dependent; the committed snapshot exists so CI catches a relative regression (e.g. an accidental full-buffer in the holdback path). Re-generate with
npx tsx scripts/bench-ai-gateway-streaming.ts --write.