Empty output

This page explains what the Empty output rule checks, when it raises a finding, and how to set it up for a project.

Definition

Empty output (key empty_output) flags a model call that returned a blank completion. The call itself worked, but it returned no answer.

The rule checks LLM spans only, that is, spans where span.span_kind == 'llm'. Other spans, such as tool calls, get no result at all from this rule. They aren't counted as a pass or a skip.

For each LLM span, the rule fires when the completion has no characters:

size(span.output_preview) == 0

When the rule fires, it records a failed verdict. The result shows these details:

• Observed value: the length of the output, which is 0.
• Reasoning: "completion is empty".

Why it matters

A blank completion is a silent failure. The provider reports success, so error monitoring doesn't catch it. Your user still gets no answer. This rule finds those calls and links each finding to the trace that raised it.

What it requires

• LLM spans in your telemetry. The rule only checks model calls. To send traces, see Instrument your code.
• Permission to change rules in the project. To create the rule, you need permission to modify the project. For details, see Project permissions.
• No model API key. This is a deterministic check. It makes no model calls, so it doesn't add model cost.

Configuration examples

This template has no parameters. Create the rule from the template catalog, then set its name, project, and severity. For the full procedure, see Create a rule.

To find the template on the New rule page, use any of these options:

• In Search templates, enter empty_output or Empty output.
• Select the Deterministic kind chip.
• In the Category rail, select Output integrity.

On the Configure rule page, the parameters area says this template has nothing to configure. The How this rule evaluates (read-only) panel shows the precondition and the predicate from the Definition section.

Here's a typical setup for a customer support assistant:

Select Save & enable to save the rule. It starts checking new spans as soon as it's saved.

To read the findings, check the Rules column on the step list or the Checks panel on a trace. To list only the traces where this rule failed, use the Rules filter on the trace list. For more information, see Explore traces.

Related

• Output length out of bounds flags completions that are shorter or longer than limits you set.
• Truncated output flags completions that stopped at the token limit.
• Stopped by content filter flags completions that the provider's safety filter stopped.
• Refusal detected flags completions where the model declined to answer.

Next steps

• Create a rule
• Rules overview
• Explore traces