Skip to main content

​
OpenTelemetry Export (OTLP)

KnoxCall’s integration logging is aligned to OpenTelemetry standards. Every request KnoxCall proxies and every webhook it delivers can be exported, in real time, to your own observability backend over the standard OTLP/HTTP protocol — so the same event stream you see in the dashboard lands in your SIEM with no custom glue.

​
What gets exported

When an OTLP endpoint is configured, KnoxCall emits three signals:

​
Traces

One SERVER span per HTTP request through the platform, plus dedicated spans for AI Gateway calls (using the OpenTelemetry GenAI semantic conventions) and security events. Spans use the standard HTTP attributes (http.request.method, url.path, http.response.status_code) with KnoxCall-specific context under the knoxcall.* namespace. KnoxCall participates in W3C distributed tracing:
  • Inbound traceparent headers are honoured, so a request that arrives already traced continues the same trace.
  • Outbound calls — both the proxied upstream request and webhook deliveries — carry an injected traceparent, so the trace extends end-to-end through KnoxCall into your upstream APIs.

​
Logs (integration logs)

Every proxied request and every webhook delivery is emitted as an OTLP LogRecord: Severity maps from outcome: 2xx/3xx → INFO, 4xx → WARN, 5xx or transport error → ERROR.
Log records carry the structured envelope (method, path, status, IDs, latency) — not request or response bodies. Full payloads remain (obfuscated) in the API Logs store, so the OTLP stream stays cheap to ship and index.

​
Metrics (analytics)

Per-request analytics are exported as OTLP metrics on a 60-second interval, so request volume, error rate, and latency show up as native dashboards/monitors in your backend: On the operator export these carry a knoxcall.tenant_id dimension; on a per-tenant export (below) they don’t, since every point already belongs to that tenant.

​
Configuring the endpoint

There are two ways to point KnoxCall at a collector. Environment variables take precedence when set; otherwise the admin-UI integration is used.
  1. Go to Settings → Integrations and switch to the Global scope (admins only; available on both cloud and self-hosted deployments).
  2. Open OpenTelemetry export.
  3. Fill in:
    • OTLP HTTP endpoint — the base URL of your collector, e.g. https://otlp.your-collector.com. KnoxCall appends /v1/traces, /v1/logs, and /v1/metrics.
    • Service name (optional) — the resource service.name (defaults to knoxcall).
    • Auth headers (optional, encrypted at rest) — comma-separated key=value pairs, e.g. Authorization=Bearer <token>, X-Scope-OrgID=acme.
    • Export traces / logs / metrics — per-signal toggles (all on by default); turn off any signal you don’t want shipped.
  4. Save. Export activates immediately — no restart required.

​
Option 2 — Environment variables

When neither an env endpoint nor the integration is configured, export is fully off and adds no overhead.

​
Compatible backends

Any OTLP/HTTP-compatible collector or backend, including:
  • Grafana (Tempo for traces, Loki for logs) / Grafana Cloud
  • Datadog (OTLP intake)
  • Elastic APM
  • SigNoz
  • Splunk (via the OpenTelemetry Collector)
  • A self-run OpenTelemetry Collector that fans out to anything downstream

​
Operator export vs per-tenant export

There are two independent OTLP destinations, for two different audiences:
  • Operator export (this page’s OpenTelemetry (OTLP export) card, Global scope, or the env vars). Ships the whole gateway’s traces + logs + metrics to the operator’s collector — for whoever runs the deployment to monitor KnoxCall itself. Operator metrics/logs carry a knoxcall.tenant_id dimension so you can facet by tenant.
  • Per-tenant export (the Telemetry export (OTLP) integration, tenant scope). Each tenant points KnoxCall at their own collector/SIEM and receives only their own request logs and analytics metrics — nothing from other tenants, and no traces. This is configured per workspace, right next to Email/Twilio/S3 in Settings → Integrations.
Both can run at once: a request fans out to the operator’s collector and the originating tenant’s collector.

​
Configure per-tenant export

  1. As a workspace admin, go to Settings → Integrations → Telemetry export (OTLP).
  2. Set the OTLP HTTP endpoint (your collector — e.g. a Datadog Agent or OTel Collector), optional service name, and optional auth headers (encrypted at rest).
  3. Save. Only that workspace’s knoxcall.proxy.request logs and knoxcall.proxy.requests / knoxcall.proxy.request.duration metrics ship to it; applies on the next request, no restart.

​
Error tracking (Sentry)

The OTLP signals above are for shipping telemetry to your own collector. To catch errors — with native grouping, alerting, and release health — KnoxCall ships an optional Sentry integration covering both tiers:
  • Browser (@sentry/react) — admin-UI uncaught exceptions, React render crashes, and unhandled promise rejections.
  • Operator console (@sentry/react, separate DSN) — the same for the console SPA. Its own project on purpose: console events identify a KnoxCall operator and carry no tenant_id, so it can be switched on independently of the tenant-facing admin UI, and its issue stream kept behind tighter access.
  • Backend (@sentry/node) — server-side exceptions on any 5xx, plus uncaught exceptions and unhandled rejections outside the request cycle, in every process: the API server, the standalone proxy, the workflow workers and each scheduled job. Events carry a component tag naming which one (api, proxy, workflow-worker, cron:retention-purge, …), so a crash in a 03:00 job is as visible as one on the request path.
It’s a separate integration card — Error tracking (Sentry) — independent of the OpenTelemetry export above. Configure either, both, or neither.

​
Configure

  1. Create Sentry project(s) and copy the DSN(s) (https://<key>@o0.ingest.sentry.io/0). A DSN identifies a project, not a secret. Using separate browser and server projects is recommended (cleaner source maps, releases, and access control), but one project for both works too.
  2. In Settings → Integrations (Global scope) → Error tracking (Sentry):
    • Sentry DSN — browser (admin UI errors) → the browser project DSN.
    • Sentry DSN — server (backend errors) → the backend project DSN.
    • Sentry DSN — operator console → the console project DSN (optional; a separate project is recommended, see above).
    • Save. (Or set SENTRY_DSN / SENTRY_DSN_BACKEND env vars — env wins, same as the OTLP endpoint. SENTRY_ENVIRONMENT sets the environment tag.)
  3. The browser DSNs are delivered to their SPA shells at page-load time (no rebuild) — the admin DSN only to the admin UI, the console DSN only to console.knoxcall.com. The backend DSN stays server-side and applies immediately on save.
When a DSN isn’t set, that SDK never initialises and adds no overhead.

​
What it captures, and what it deliberately doesn’t

  • Captures: browser uncaught errors / rejections / React render errors (with the component stack); backend 5xx exceptions and process-level crashes — tagged with release and environment. Backend errors carry the opaque errorId that’s also in the server log and the client response, for correlation.
  • Tenant-filterable: events are tagged with tenant_id (and identified by user UUID only — never email/name) on both tiers, so you can filter “all errors for tenant X” in Sentry. These are internal identifiers, not PII, but they are a data category sent to Sentry — see the note below.
  • No session replay. Replay records the live admin DOM, which on a multi-tenant console can capture other tenants’ data — off by design.
  • No PII. Cookies, IPs, request bodies and URL query parameters are never attached — every Sentry SDK sets dataCollection explicitly to collect none of them, keeping only the browser’s User-Agent. Token/secret-shaped values are redacted, and query strings and URL fragments stripped, from event messages, breadcrumbs and page URLs before an event leaves the process or the browser; the backend reports the request path only.
  • Optional: warn+ logs. Set SENTRY_LOGS=1 to ship warn and error lines from the application logger to Sentry as structured logs, attached alongside the exceptions. Worth it because the backend SDK carries no breadcrumbs by design, so a stack trace otherwise arrives with no run-up. Only text that has already been through the redaction grammar is sent — Sentry’s consoleLoggingIntegration is deliberately not used, as it would ship un-redacted console.* output. Off by default: log ingest is billed by volume, and it widens what leaves the process from exceptions to log bodies.
  • No tracing interference. The backend SDK runs as a pure error reporter (no Sentry OpenTelemetry setup, no loader hook, no auto-instrumentation) — OpenTelemetry above keeps owning all traces and propagation. (Backend errors are therefore not OTel-trace-linked; the traces live in your OTLP backend.)

​
Browser source maps

Without source maps a browser stack trace is minified frames against hashed chunk names. Set SENTRY_ORG, SENTRY_PROJECT and SENTRY_AUTH_TOKEN on the machine that builds admin-ui, and the build uploads maps to Sentry and then deletes them from dist/. The console build reads the same variables, with SENTRY_PROJECT_CONSOLE overriding SENTRY_PROJECT when the two SPAs report to separate projects. These are build-time variables, not runtime config — they do not belong in the server env file. A build without all three emits no maps at all. That is deliberate: an un-uploaded map left in dist/ would serve readable admin-console source from the admin origin, which is a worse outcome than unsymbolicated traces.

​
Sentry as an OTLP backend

You can point the OTLP HTTP endpoint at Sentry’s own OTLP ingest, so spans and integration logs land in the same project as your errors. Copy the values from Project Settings → OpenTelemetry in Sentry: Give the endpoint without a signal suffix — KnoxCall appends /v1/traces and /v1/logs itself, which is exactly what Sentry expects.
Sentry does not ingest OTLP metrics. Send them somewhere else, or turn them off — otherwise the metric exporter POSTs to a path Sentry rejects every 60 seconds for as long as the process runs. Traces and logs are unaffected.

​
Sending one signal somewhere else

Each signal can override the endpoint above, using the standard OpenTelemetry variable names:
Two rules, both from the OTel spec and both deliberate:
  • A signal endpoint is used verbatim — it already carries its path, so give the full URL ending in /v1/metrics. Nothing is appended.
  • Signal headers replace the generic ones, they do not merge. That is what keeps your Sentry key out of requests to the metrics backend, and the metrics backend’s token out of requests to Sentry.
The same works for TRACES and LOGS. Signals with no override keep using the endpoint and headers above. These are env-only; the Integrations tab keeps a single endpoint field, since splitting signals across vendors is an operator concern rather than a self-host one. To simply turn metrics off instead, untick Export metrics or set OTEL_METRICS_EXPORTER=none. Two things this does not replace. Sentry’s OTLP ingest is in open beta and is thinner on grouping and breadcrumbs than the native SDK, so keep the Sentry DSN — server field set for errors. And OTLP retention is a Sentry plan setting measured in weeks, so it is a place to alert and triage from, not a system of record — audit evidence belongs in the audit log and its offsite archive.
Configuring a Sentry DSN sends error reports to Sentry. For the hosted service that makes Sentry a sub-processor; if you run KnoxCall yourself the data goes to your Sentry project. Either way, review your sub-processor disclosures before enabling it in production.
  • API Request Logs — full request/response history in the dashboard
  • Audit Logs — security and configuration events (also deliverable via the audit.event webhook)
  • Webhooks — push individual events to your own endpoints