Output length out of bounds

This page explains what the Output length out of bounds rule checks, when it raises a finding, and how to set its length limits.

Definition

Output length out of bounds (key output_length_bounds) flags a model completion that's shorter or longer than you expect. It catches truncated answers and runaway generation without a judge model.

The rule checks LLM spans only. It fails a span when the completion has fewer characters than Minimum characters or more characters than Maximum characters. Otherwise, the span passes.

When a span fails, the result records:

• The observed length of the completion.
• An excerpt of the output.
• A summary sentence, for example "completion is 5210 characters, expected between 1 and 4000".

The summary appears in the Check results popover and in the Checks panel on the step's detail page. To find these results, see Explore traces.

Why it matters

A completion can come back with a success status and still be wrong in shape. A very short answer often means the model stopped early. A very long one often means it kept generating past the useful answer. This rule finds both cases with a simple length check, so you don't need to review each completion.

Because it's a deterministic check, it doesn't call a model to decide.

What it requires

• Your application sends traces that contain LLM spans. To set this up, see Instrument your code.
• You have permission to change rules in the project. For details, see Project permissions.

The rule has two parameters. Both are required.

If a value isn't valid, the form shows a message after you try to save, for example "Minimum characters must be a whole number." or "Maximum characters must be at least 1."

Configuration examples

1. On the New rule page, enter output_length_bounds in Search templates, and then select Configure › on the Output length out of bounds card.

2. On the Configure rule page, enter values for Minimum characters and Maximum characters, and then select Save & enable.

For the full procedure, see Create a rule.

The following table shows common settings.

Related

• Empty output flags a completion that's blank.
• Truncated output flags a completion that stopped at the token limit.
• LLM call over token budget flags a single model call that used too many tokens.

Next steps

• Create a rule
• Rules overview
• Explore traces