Redact by pattern

This page shows you how to replace text that matches your own regular expressions with a placeholder before telemetry leaves your application, and how to let chosen matches through.

Prerequisites

• Your application sends traces with the Python SDK. For setup, see Instrument your code.

Scrub text that matches a pattern

Pattern redaction is set in code only. You can't set patterns with environment variables. The SDK doesn't include any built-in patterns, so you write your own.

1. Write each pattern as a Python regular expression. Use a string or a compiled re.Pattern object.
2. Pass the patterns to aio.redaction_policy(patterns=[...]).
3. Pass the policy to aio.init(redaction=...). Call aio.init once, at startup, before you create any HTTP or model clients.
4. Run your application and send a request that contains text matching one of your patterns.

import re
import acceldata_aio_tracer as aio
policy = aio.redaction_policy(
    patterns=[
        # Email addresses
        r"[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}",
        # Keys that look like sk-XXXXXXXXXXXXXXXXXXXX
        re.compile(r"sk-[A-Za-z0-9]{20,}"),
    ],
)
aio.init(redaction=policy)

Add any other aio.init arguments your application already uses.

Each match becomes __REDACTED__. For example, Contact jane.doe@contoso.com about the refund is recorded as Contact __REDACTED__ about the refund. The SDK never writes the matched text to a log.

Patterns run one after another, in the order you list them. Each pattern runs on the output of the one before it.

The SDK compiles string patterns when you call aio.redaction_policy(...). An invalid pattern raises re.error right then, at startup, so you find out before any data is sent.

Let specific matches through

To keep some matches, pass an allow function. It receives each matched text as a str. Return True to keep the match, or False to replace it with __REDACTED__.

import acceldata_aio_tracer as aio
def allow(match: str) -> bool:
    # Keep your support address. Redact every other email address.
    return match == "support@example.com"
policy = aio.redaction_policy(
    patterns=[r"[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}"],
    allow=allow,
)
aio.init(redaction=policy)

If allow raises an error, the SDK withholds the match.

allow only works with patterns. If you set allow without any patterns, nothing is redacted.

Know what patterns scan

Patterns scan this content:

• The text parts of input and output messages. They run after any tool arguments are withheld.
• Tool call arguments and tool call results, when they're plain strings that haven't already been replaced by a fingerprint.

Patterns don't scan this content:

• System instructions and tool definitions. To withhold these, see Hide system prompts and tool definitions.
• The arguments of structured tool calls inside messages. To withhold these, see Redact tool arguments and results.
• Any other attribute. To withhold one, see Drop attributes and spans.

Check the result in the UI

1. Open the trace in AIO and select the step that handled the matching text. For help finding a trace, see Explore traces.
2. In the step pane, go to the Input / Output tab.

The Input and Output previews show scrubbed text as __REDACTED__.

Troubleshooting

• Your application fails at startup with re.error. One of your string patterns isn't a valid regular expression. Fix the pattern and start the application again.
• Matching text still appears in a trace. Check that the text is in content that patterns scan. Also check that your allow function doesn't return True for it.

Next steps

• Redaction overview
• Redact tool arguments and results
• Drop attributes and spans
• Python SDK