Redact sensitive data from AI agent traces

Prerequisites

Before changing the trace pipeline, identify the collection boundary through which every agent span passes before export. You need access to that boundary’s configuration and to the instrumentation that creates agent, model-call, and tool spans. Also identify every service to which trace context is propagated; testing only the first service will not test baggage exposure downstream.

Redact at the collection boundary

1. Inventory every place that can add content to a trace.

   Follow one agent run from its first span through model calls, tool calls, errors, and export. Record which components create span attributes, events, error details, and baggage. Inspect actual emitted telemetry rather than relying on the intended schema: instrumentation can add fields that the agent application did not create directly.

   Mark credentials, API keys, personal data, and full tool request or response payloads as prohibited. The OWASP Logging Cheat Sheet’s data-exclusion guidance, retrieved 2026-08-26, says access tokens, passwords, encryption keys, sensitive personal data, and other primary secrets should usually be removed, masked, sanitized, hashed, or encrypted rather than recorded directly. For agent traces, removal is the safer default when the value is not needed to operate the system.

   Do not treat this inventory as the filter itself. It tells you what existing instrumentation emits and where to stop generating sensitive fields, but a list of known bad fields becomes incomplete as soon as a tool or instrumentation library adds another name.

2. Define the small trace schema you intend to keep.

   Start with the fields needed to answer operational questions without reconstructing sensitive content:

   Operational need	Retained trace data	Content deliberately omitted
   Join work from one run	Correlation identifiers	User data used as an identifier
   Identify the attempted operation	Tool name	Tool arguments and full payloads
   Find slow operations	Start time, end time, or duration	Request and response content
   Separate success from failure	Status	Raw failure payloads
   Group failures	Redacted error class	Error text containing inputs, outputs, or credentials

   Keep field names and types fixed. A correlation identifier should identify the run or operation, not encode the user, prompt, or tool input. If several services must agree on those identifiers, define that contract alongside your correlation identifiers across agent runs, model calls, and tools.

   This schema gives up ad hoc inspection of arbitrary attributes. That is the intended cost: engineers must add a newly required operational field deliberately instead of receiving every new field by default.

3. Enforce an allowlist before the exporter.

   Configure the collection boundary to delete every trace attribute except the approved schema. OpenTelemetry’s sensitive-data guidance, last modified 2026-01-14 and checked 2026-08-26, recommends collecting only data that serves an observability purpose and documents a Collector redaction processor that deletes attributes not present in an allowed-attributes list.

   Prefer this allowlist to a blocklist. A blocklist can remove only credential and personal-data fields whose names you already know. It misses a new field with an unexpected name, including one introduced by a new tool, a changed payload shape, or updated instrumentation. An allowlist drops that field until someone reviews and approves it.

   Remove sensitive values in instrumentation too when you control it, but keep the boundary rule. Source-side removal reduces what a component emits; the boundary rule supplies one enforceable policy for all components. Put the rule before every exporter so no normal export path can bypass it.

4. Give baggage a narrower policy than stored span attributes.

   Do not put credentials, API keys, personal data, or full tool payloads in trace baggage. OpenTelemetry’s context-propagation documentation, last modified 2026-08-10 and checked 2026-08-26, states that baggage carries arbitrary key-value pairs across service boundaries and warns that sensitive values may be logged or sent to untrusted downstream services.

   Treat baggage as a propagation mechanism, not spare trace storage. If a service needs a correlation identifier to join work across the run, allow that identifier explicitly. Tool arguments, model content, tool results, credentials, API keys, and personal data do not belong there. A value that seems harmless in the originating process can cross into services with different logging behavior.

   Filtering exported spans alone is insufficient because the baggage has already travelled with the request. Apply the baggage allowlist when creating outgoing context, and sanitize or reject unapproved incoming baggage before forwarding it again.

5. Replace payload detail with operational classifications.

   For each tool call, emit the tool name, timing, and status. On failure, map the failure to a redacted error class chosen by the application; do not copy the full tool response or an unrestricted error message into that class. The class must say what kind of operation failed without preserving the content that caused it.

   Apply the same rule to successful calls. Success does not make a payload safe: a successful tool response can still contain credentials or personal data. Record the successful status and duration, not the full response.

   This makes traces unsuitable as a complete transcript. If the team needs content to reconstruct what an agent did after the fact, design that as a separate, explicit data path with its own access and agent-data retention boundaries. Do not restore full payloads to traces merely to make replay convenient.

6. Test the exported trace and every propagation hop.

   Run a synthetic agent task containing unmistakable dummy values representing a credential, an API key, personal data, and a full tool payload. Exercise both successful and failing tool calls. Then inspect the telemetry after the collection boundary and the context received by each downstream service.

   The prohibited values must be absent from span attributes, events, error details, and baggage. The same trace must still contain its correlation identifiers, tool names, timing, status, and redacted error classes. Searching only the trace viewer is not enough for baggage: inspect what downstream services receive and any telemetry they emit.

   Add one unapproved attribute name to the synthetic instrumentation and repeat the test. It must disappear without a filter update. That check proves the pipeline is operating as an allowlist rather than as a blocklist of the sensitive names known today.

7. Make schema changes reviewable.

   Store the allowlist with the trace-pipeline configuration and require a review when adding a field. The review question is concrete: which operational decision requires this value, and can a classification, identifier, status, or timing value answer it without retaining content? Re-run the synthetic export and propagation tests after each approved change.

Expected result

A completed implementation exports traces that can join an agent run across services, show which tools ran, measure their timing, distinguish statuses, and group failures by redacted class. Credentials, API keys, personal data, full tool payloads, and unapproved new attributes are absent from exported telemetry and propagated baggage.