Groundedness

This page explains what the Groundedness rule checks, what it needs to run, and how to set its threshold.

Definition

Groundedness checks whether a trace's answer is supported by the documents your application retrieved for it. A judge model reads the answer and the retrieved documents and gives the trace a score from 0.0 to 1.0. When the score is below your threshold, the rule records a failure.

The rule checks only traces that have output. Traces with empty output aren't evaluated, so they get no result at all: not a pass and not a failure.

For each trace it checks, the rule records one of these verdicts:

• Fail: the judge's score is below Minimum acceptable score.
• Pass: the score is at or above Minimum acceptable score.
• Error: the check couldn't decide, for example because the judge couldn't be reached. An error doesn't count as a pass.

Because Groundedness is a numeric rule, the score appears with its verdict in the Check results popover, rounded to two decimals. In the Checks panel on a trace, each result also shows the judge model and the prompt version, groundedness@v1. The judge prompt handles the answer and the documents as data to score, not as instructions to follow.

Why it matters

An answer can sound confident and still say things the source documents don't support. Groundedness flags those answers so you can find the traces where your retrieval workflow made claims its sources don't back up.

What it requires

• Retrieved documents in your telemetry. The judge compares the answer with the documents recorded on the trace. By default, it reads up to 12 retrieved documents per trace and up to 2,000 characters of each.
• A judge model. Your AIO deployment must have a model API key set up for judge rules. Without one, Groundedness never runs, and the Rules column shows its traces as "pending".
• Permission to change rules in the project. For details, see Project permissions.

Configuration examples

To create the rule, open the New rule catalog. Search for groundedness, or select the Judge chip. Then select Configure › on the Groundedness card.

On the Configure rule page, set the parameters described in the following table.

The How this rule evaluates (read-only) section shows when the rule applies and when it fires. Under Judge prompt (read-only · version), select Show full prompt › to read the prompt the judge uses. The prompt comes with the template, and you can't edit it.

Choose a threshold based on how strict you want the rule to be:

If you leave a required parameter empty or enter a value outside its range, you see a message such as "Minimum acceptable score is required." or "Minimum acceptable score cannot exceed 1."

Select Save & enable to save the rule and turn it on. For the full procedure, see Create a rule.

Related

• Missing citation checks whether a grounded answer includes a citation marker. It doesn't call a model.
• Agent answered without using a tool flags traces that answered from memory instead of calling a tool.

Next steps

• Create a rule
• Rules overview
• Explore traces
• Project permissions