Patterns

Patterns let you automatically detect specific behaviors in your agent’s interactions. The platform scans every conversational run and flags matches, so you can catch issues without manually reviewing traces.

​

Built-in patterns

Three patterns are available out of the box:

• Frustration - the user expresses annoyance, dissatisfaction, or negative sentiment toward the agent
• Confusion - the user indicates they don’t understand or are lost
• Task failure reported - the user reports that the agent didn’t complete their request correctly

Built-in patterns cannot be edited or deleted.

​

Custom patterns

Create your own patterns to monitor behaviors specific to your use case. Custom patterns use AI-powered classification to detect matches based on your description and optional labeled examples.

​

Creating a pattern

Click Create pattern to open the creation wizard, which has three steps:

​

Step 1: Define the pattern

• Pattern name - a human-readable name (up to 50 characters). Example: “User frustration”
• Unique pattern ID - an identifier auto-generated from the name in kebab-case (up to 50 characters). Example: “user-frustration”
• Description - describe what this pattern detects (up to 350 characters). The more specific your description, the better the detection accuracy.

Trigger conditions

Control which runs are checked for this pattern:

• When to check - choose one of:
  • Check all runs (default)
  • Check runs that have any of specific tools
  • Check runs that have all of specific tools
• Metadata filters (optional) - only check runs that match specific metadata key-value pairs. You can add up to 10 filters.

​

Step 2: Label examples (optional)

Improve detection accuracy by labeling recent runs as positive or negative examples:

• Browse runs from the past 7 days
• For each run, mark it as matches (positive) or does not match (negative)
• You can label up to 100 examples
• Filter runs by text search, run type, attributes, or metadata

​

Step 3: Configure alerts

• Enable Slack alerts - toggle whether to send notifications when this pattern is detected
• Select channels - choose which Slack channels receive alerts (requires a connected Slack workspace with subscribed channels)
• Alert frequency - choose when to send alerts:
  • Every detection
  • When a threshold is exceeded (e.g., 5 detections within 60 minutes)
  • Daily digest

​

Patterns list

The patterns page shows all configured patterns - both built-in and custom - with their detection counts. Use the search bar to filter by name.

​

Pattern detail

Click a pattern to see all runs where it was detected. Each run shows:

• The confidence score of the detection
• Evidence explaining why the pattern matched
• A link to the full run detail

For custom patterns, you can edit trigger conditions and alerts, or delete the pattern.