Conversation ran long

This page explains what the Conversation ran long rule checks, when it raises a finding, and how to set its turn limit.

Definition

Conversation ran long (key session_turn_count) flags a conversation that takes more turns than you expect.

The rule checks only conversations with more than one turn. A single-turn conversation gets no result, so it isn't counted as a pass or a failure.

The rule fails a conversation when its number of turns is greater than Expected turns. For example, with Expected turns set to 10, a conversation of 10 turns passes and a conversation of 11 turns fails.

When the rule fails, the result's reasoning gives the turn count and the limit, for example: "conversation ran 14 turns, expected at most 10".

The check can run again as a conversation grows. The newer result replaces the earlier one, so a conversation that passed at 8 turns can fail later if it goes past the limit.

Why it matters

A long conversation often means the user didn't get what they needed and kept asking. You can't see that from any single turn, because each turn can look fine on its own. Counting the turns shows you that pattern across the whole conversation.

What it requires

• Sessions in your telemetry. The rule evaluates conversations, so your traces need session identifiers. To set them up, see Track users and sessions.
• Permission to change rules in the project. To create or edit this rule, you need permission to modify the project. For details, see Project permissions.

This rule is deterministic, so it doesn't call a model.

Configuration examples

The rule has one parameter.

Expected turns is the highest number of turns a conversation can have before it fails. The template suggests 5, 10, and 25 as example values:

• 5: Flags any conversation that runs past five turns.
• 10: The default. Flags conversations of 11 turns or more.
• 25: Flags only conversations that run much longer than usual.

To add the rule to a project:

1. On the New rule page, enter Conversation ran long in Search templates, or select Agent behaviour in the Category list.

2. On the Conversation ran long card, select Configure ›.
3. On the Configure rule page, for Expected turns, enter the number of turns you allow.

4. Select Save & enable.

If the value isn't valid, the page shows one of these messages after you try to save:

• "Expected turns is required."
• "Expected turns must be a whole number."
• "Expected turns must be at least 2."

For the other fields on the form, such as Name, Project, and Severity, see Create a rule.

Related

• Conversation had failed turns flags conversations where one or more turns errored.
• Conversation over token budget flags conversations that used more tokens than their budget.

Next steps

• Create a rule
• Rules overview
• Explore traces
• Track users and sessions