> ## Documentation Index
> Fetch the complete documentation index at: https://docs.knoxcall.com/llms.txt
> Use this file to discover all available pages before exploring further.
# Sealed state and unseal
> What happens when KnoxCall can't reach your KMS, and how to restore service. The sealed state is reversible — revocation is not.
# Sealed State and Unseal
When KnoxCall can't unwrap the tenant master key from your KMS, the tenant enters the **sealed state**. New agent sessions cannot be issued until access is restored.
This is a safety mechanism, not a failure. It guarantees that your kill switch works: if you revoke KnoxCall's IAM access, the tenant seals within one cache TTL.
## What triggers sealing
* IAM permission revoked (your kill switch in action)
* KMS key disabled or scheduled for deletion
* Network partition between KnoxCall and your KMS endpoint
* KMS service outage in your cloud provider's region
* KMS key version revoked (this triggers **permanent** sealing — see below)
## What happens when sealed
Sealing is enforced at the **agent session layer**, not at individual proxy requests:
* **Session endpoint**: `POST /agent/v1/session` (the endpoint agents call to obtain a session bundle) returns `503 Service Unavailable` with `{ "error": { "type": "tenant_sealed", "message": "..." } }`
* **In-flight agents**: agents that already hold a valid session continue to operate normally until their session expires — at most **5 minutes** for BYOK tenants
* **After session expiry**: agents must renew; renewal fails with 503 until the tenant is unsealed
* **Admin UI**: a sealed banner appears on the Customer KMS page (Settings → Security → Tenant Master Key → Customer KMS) with the last KMS error message
* **KMS config row**: `sealed_since` timestamp set; visible in `GET /admin/tenant-kms`. No audit action is written for the seal event itself — check `last_verify_error` for the reason
There is no per-request `423 Locked` response at the proxy layer. The sealed check happens once per session renewal. An agent mid-request is unaffected until its current session TTL runs out.
## Sealed vs revoked — a critical distinction
| | Sealed | Revoked |
| ----------- | ---------------------------------------- | -------------------------------------------------------- |
| Trigger | KMS unreachable or IAM revoked | Tenant master key version explicitly revoked in KnoxCall |
| Reversible? | ✅ Yes — restore KMS access | ❌ No — permanent cryptographic erasure |
| Data state | Intact at rest, temporarily inaccessible | Data wrapped by that version is permanently unreadable |
| Recovery | Restore IAM + unseal | None |
Revoking the master key in KnoxCall is different from revoking IAM access. If you want a recoverable kill switch, revoke the IAM grant. If you want data destruction, revoke the master key in KnoxCall.
## Monitoring for seal events
**Seal state**: Sealing records `sealed_since` and `last_verify_error` on the tenant's KMS config row. There is no `byok.kms.seal` audit action — the seal itself is not logged in the audit chain. Check `GET /admin/tenant-kms` and look for a non-null `sealed_since` field.
**Unseal audit log**: When the tenant unseals (manually or automatically), a `byok.kms.unseal` critical audit entry is written. Filter by that action in Monitoring → Audit Logs to see the history of seal/unseal cycles.
**Alerts**: There is no dedicated seal webhook event, so monitor by polling `GET /admin/tenant-kms`: if `sealed_since` becomes non-null, page on-call immediately — new agent sessions will fail within the next 5 minutes.
## Unsealing
### Automatic unseal
Once you restore KMS access (re-grant the IAM permission, re-enable the key, resolve the network partition), the tenant clears its sealed flag **lazily, on the next successful unwrap** — no manual action required. There is no background poller: the sealed flag is cleared the next time KnoxCall has to unwrap that tenant's master key (the next cache miss), which is driven by agent traffic rather than a fixed timer. If no traffic arrives, use the manual unseal endpoint below to force it immediately.
The `byok.kms.unseal` action is logged when the tenant comes back up.
### Manual unseal via API
If you need to force an immediate unseal attempt (rather than waiting for the next unwrap), call the unseal endpoint. This is an admin (control-plane) endpoint: it's served on the admin host, authenticated with your session JWT plus the `X-Tenant-ID` header, and requires the owner/admin role. It also requires a **recent step-up verification within a 2-minute window**:
```bash theme={"dark"}
curl -X POST https://admin.knoxcall.com/admin/tenant-kms/unseal \
-H "Authorization: Bearer $KC_ADMIN_JWT" \
-H "X-Tenant-ID: $KC_TENANT_ID"
```
The step-up requirement is enforced server-side against your account, not via a header on this request. Complete a fresh step-up check (passkey or 2FA) in the dashboard — **Settings → Security**, or any step-up prompt — within the last 2 minutes before calling unseal; the window is intentionally tight. If no recent verification is on file, the endpoint returns `403` with `requires_step_up: true`.
This endpoint is idempotent — calling it when the tenant is already unsealed returns `409 not_sealed`.
Response on success:
```json theme={"dark"}
{ "unsealed": true }
```
If the KMS is still unreachable or access is still denied, the response is `502 unseal_failed` with a `reason` field:
```json theme={"dark"}
{
"error": "unseal_failed",
"reason": "access_denied",
"message": "..."
}
```
`reason` is either `"access_denied"` (IAM grant still missing or invalid) or `"unreachable"` (network / KMS service issue).
### Unseal via admin UI
1. Go to **Settings → Security → Tenant Master Key → Customer KMS**
2. The sealed banner shows the last KMS error
3. After restoring KMS access, click **Unseal**
## Troubleshooting
**Unseal stuck, KMS access looks correct:**
* Verify the IAM role still exists and the trust policy is intact: `aws iam get-role --role-name KnoxCallKMSAccess`
* For GCP: verify KnoxCall's principal still has `roles/iam.serviceAccountTokenCreator` on your service account — run `gcloud iam service-accounts get-iam-policy knoxcall-kms-sa@YOUR_PROJECT.iam.gserviceaccount.com`
* For Azure: check the `Key Vault Crypto User` role assignment still exists on the vault
**`502 unseal_failed` with `reason: "access_denied"`:**
* The IAM/RBAC grant was removed or expired — restore it and retry immediately
* For Azure: confirm the client secret env var is still set on the KnoxCall server process and hasn't rotated
**`409 not_sealed` on the unseal endpoint:**
The tenant is not currently sealed. No action needed — the tenant is operating normally.
**Tenant unseals but reseals again after a few minutes:**
* The IAM grant may have been partially restored — check that both wrap and unwrap operations are permitted (AWS: `kms:Encrypt` and `kms:Decrypt`; Azure: `wrapKey` and `unwrapKey`)
* KMS key rotation in your cloud provider may have invalidated the key reference stored in KnoxCall — check `GET /admin/tenant-kms` for `last_verify_error`
Rotate your tenant master key to a new version
Unseal and rotate endpoint reference
Re-check your IAM setup
Find seal and unseal events in the audit trail