> ## Documentation Index
> Fetch the complete documentation index at: https://docs.knoxcall.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Streaming Guarantees

> Three streaming PII-redaction modes — holdback (mid-stream), buffer (full-buffer), monitor (passthrough) — with a published, CI-diffed latency benchmark. Mid-stream redaction adds sub-millisecond overhead.

# Streaming Guarantees

Most gateways can only inspect a streamed AI response *after* it finishes, or *log* what they saw. KnoxCall redacts **inside the live SSE stream**. This page states the per-mode guarantee and publishes the latency cost — a number competitors who buffer or don't enforce mid-stream can't put on the table.

## The three modes

Each agent picks a `pii_streaming_mode` (default `holdback`).

| Mode                   | Client-visible guarantee                                                                                                                                                                                       | First-byte latency                                     | Use when                                                                            |
| ---------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------ | ----------------------------------------------------------------------------------- |
| **holdback** (default) | PII is redacted **inside the stream** using a sliding hold-back tail (`pii_streaming_holdback_chars`, default 96) so an entity that straddles two SSE frames is still caught before any part of it is emitted. | Near-instant — tokens flow as they arrive.             | The default. Real-time UX with real protection.                                     |
| **buffer**             | The **entire** response is accumulated, redacted once, then emitted. Zero chance a cross-frame entity slips through, at the cost of first-byte latency (the client waits for the whole message).               | Deferred — first byte after the full response arrives. | Maximum assurance; non-interactive/batch calls.                                     |
| **monitor**            | The stream passes through **unredacted** for lowest latency, but every entity is still detected and written to the [PII event ledger](/ai-gateway/pii-redaction) for alerting and evidence.                    | Passthrough — no added first-byte latency.             | You need visibility, not enforcement (e.g. observing before you turn redaction on). |

All three record detections; only `holdback` and `buffer` alter the bytes the client sees.

## Why holdback is the hard one (and the differentiator)

A sensitive token can arrive split across SSE frames — `"...SSN 123-"` in one delta, `"45-6789..."` in the next. A naïve mid-stream redactor emits the first half before it can tell it's PII. KnoxCall's hold-back FSM keeps the last N characters buffered until enough context has arrived to decide, then splices any redaction back into the SSE frame and re-serialises it — so the client never sees a partial entity, and framing is never corrupted. Bedrock punts (`stream:false` required); Cloudflare buffers; Portkey's streaming guardrails are log-only. KnoxCall enforces.

## Published benchmark

`scripts/bench-ai-gateway-streaming.ts` drives a representative Anthropic response (120 text deltas seeded with email/SSN/credit-card entities) through each mode and a raw passthrough baseline, and records **p50/p95 TTFB** (time-to-first-emitted-byte) and **total drain time**. The snapshot lives at `scripts/bench-ai-gateway-streaming.results.json` and CI diffs it, so a regression in the flagship path fails the build.

This is a CPU-bound micro-benchmark of the redaction Transform itself (no network) — it isolates the overhead the gateway *adds* on top of the hundreds of milliseconds an LLM already takes. Representative numbers on commodity CI hardware:

| Mode         | TTFB p50      | Total p50 | TTFB overhead vs passthrough |
| ------------ | ------------- | --------- | ---------------------------- |
| passthrough  | \~0.02 ms     | \~0.05 ms | —                            |
| **holdback** | **\~0.04 ms** | \~1.4 ms  | **\~0.02 ms**                |
| buffer       | \~0.28 ms     | \~0.42 ms | \~0.26 ms                    |
| monitor      | \~0.01 ms     | \~0.34 ms | \~0.00 ms                    |

The headline: **mid-stream redaction (holdback) adds sub-millisecond first-byte latency** — noise against LLM response times measured in hundreds of milliseconds. You get real, enforced, mid-stream PII protection essentially for free on the latency budget.

<Note>
  Absolute numbers are hardware-dependent; the committed snapshot exists so CI catches a *relative* regression (e.g. an accidental full-buffer in the holdback path). Re-generate with `npx tsx scripts/bench-ai-gateway-streaming.ts --write`.
</Note>

## Configure the mode

Set it on the agent — via the **PII tab** on the gateway detail page, or over the API:

```bash theme={"dark"}
curl -X PATCH https://api.knoxcall.com/v1/ai-gateway/agents/<agentId> \
  -H "Authorization: Bearer tk_live_..." \
  -H "content-type: application/json" \
  -d '{"pii_streaming_mode": "buffer"}'
```