Conversation had failed turns

This page explains what the Conversation had failed turns rule checks, when it raises a finding, and how to set the number of failed turns it tolerates.

Definition

Conversation had failed turns (key session_error_turns) checks each conversation for turns that ended in an error. It raises a finding when the number of failed turns is above the number you tolerate.

A deterministic rule doesn't call a model. It compares the count of failed turns with your Tolerated failed turns value. When the count is higher, the conversation fails the check. Otherwise, it passes.

When the rule fails a conversation, the result's reasoning tells you how many turns failed out of the total, for example "2 of 7 turns errored".

Why it matters

A failed turn is easy to miss. When a user retries and the next turn succeeds, per-span monitoring no longer shows that anything went wrong. This rule looks at the whole conversation, so you still see that a turn failed even after the user got an answer.

To catch failures inside a single request instead, use Trace contains failed spans.

What it requires

• Conversations in your telemetry. The rule evaluates sessions, so your traces need session identifiers to group into conversations. To set them up, see Track users and sessions.
• Permission to change rules. To create the rule, you need permission to modify the project it watches. For details, see Project permissions.

Configuration examples

The rule has one parameter.

The conversation fails when the number of failed turns is greater than this value. The following table shows common settings.

To add the rule to a project:

1. On the Rules page, select + New rule.
2. On the New rule page, enter Conversation had failed turns in Search templates, and then select Configure › on the card.

3. On the Configure rule page, for Project, select the project to watch.
4. For Severity, select a level. The default is MEDIUM.
5. For Tolerated failed turns, enter a whole number, such as 0, 1, or 3.

6. Select Save & enable.

The rule is enabled as soon as you save it. For the full procedure and every field on the form, see Create a rule.

Troubleshooting

If the value you enter for Tolerated failed turns isn't valid, the form shows one of the following messages when you try to save:

• "Tolerated failed turns is required."
• "Tolerated failed turns must be a whole number."
• "Tolerated failed turns must be at least 0."

Related

• Trace contains failed spans checks for failed spans in a single request.
• Conversation ran long checks for conversations with more turns than expected.
• Conversation over token budget checks the total tokens a conversation used.

Next steps

• Create a rule
• Rules overview
• Track users and sessions
• Explore traces