Stopped by content filter

This page explains what the Stopped by content filter rule checks, when it raises a finding, and how to set it up for your provider.

Definition

Stopped by content filter (key content_filter_stop) flags a large language model (LLM) call that the provider stopped partway through because its own safety filter stepped in.

The rule looks only at LLM spans. For each one, it compares the span's finish reason with the value you set in Finish reason meaning "filtered". When they're equal, the verdict is a fail. Otherwise, it's a pass.

A failed check shows a reason such as "provider stopped on its own safety filter, finish_reason content_filter".

Why it matters

A content filter stop isn't the same as a model refusal. When the provider's filter cuts off a response, your prompt and your application aren't broken. The provider stepped in. Tracking these stops separately keeps them from mixing with refusals and errors, so you can see how often the provider blocks your traffic.

To catch a model that declines to answer on its own, use Refusal detected instead.

What it requires

• LLM spans with a finish reason. The rule reads the finish reason your instrumentation records for each model call. To start sending traces, see Instrument your code.
• Permission to change rules in the project. To create this rule, you need modify permission on the project. For details, see Project permissions.

This is a deterministic check, so it doesn't call a model and doesn't need a judge model.

Configuration examples

The rule has one parameter.

The span's finish reason must equal this value exactly. Providers don't all use the same wording, so set it to the value your provider reports. The template suggests these values:

• content_filter (the default)
• safety
• blocked

Each rule matches one value. If your providers report different values, create one rule for each value.

If you leave the field blank, the form shows "Finish reason meaning "filtered" is required." when you try to save.

To set up the rule:

1. On the New rule page, enter content_filter_stop in Search templates, and then select Configure › on the Stopped by content filter card.

2. On the Configure rule page, for Finish reason meaning "filtered", enter the value your provider reports, such as safety.
3. Fill in the other fields, and then select Save & enable.

For the full description of every field on the form, see Create a rule.

Related

• Refusal detected: the model declined to answer.
• Truncated output: the provider stopped at the token limit.
• Empty output: the model returned a blank completion.

Next steps

• Create a rule
• Rules overview
• Explore traces