Where discovery shows up in the product
Discovery is a running process, not a page you visit once. It surfaces through several product surfaces:Recaps
Daily, weekly, and monthly reports that scan every run and call out the trends, anomalies, and specific interactions worth reviewing. Emerging failure themes, cost anomalies, shifting user behavior, and new topics all surface here first.
Topic clusters
Automatic grouping of conversations by what users are trying to do. Trending topics, growing use cases, and requests the agent handles poorly all appear as clusters.
Failures
Recurring errors across runs and tool calls, automatically grouped and ranked by frequency. Regressions show up as spikes in an existing failure or as a brand new group.
Built-in Patterns
Frustration, confusion, and task-failure classifiers run on every conversation by default. You do not configure them.
The discovery-to-monitoring loop
Discovery is only useful if it feeds back into the parts of the product you check regularly. The workflow:1
Read the recap
The weekly recap calls out lowlights, emerging themes, and anomalies with links to the runs behind each one.
2
Investigate
Open a linked run to see the full trace, or ask Insight Search a follow-up question like “show me every run where the refund tool returned an error but the response said the refund was processed”.
3
Track
If the finding is worth monitoring, add a custom Pattern so future runs are automatically flagged and counted.
4
Alert
Send pattern hits to Slack so regressions surface without waiting for the next recap.
What discovery finds today
Discovery is grounded in the data The Context Company already captures: prompts, responses, model calls, tool calls, tool arguments, tool results, errors, latency, tokens, cost, feedback, and metadata. Concretely, discovery surfaces:- Emerging failure themes: a specific tool starting to return errors, a workflow retrying more often, a class of prompts producing empty responses.
- New or growing topics: a use case that was rare last week and common this week (topic clusters).
- Behavioral signals: frustration, confusion, and task-failure Patterns run automatically without configuration.
- Anomalies: cost, latency, and volume spikes on a tool, model, or agent.
- Silent failures: transcripts that look successful but contain tool errors or contradictions in the execution trace.
- Regressions: metrics or Patterns changing after a release when agent version metadata is attached.
Discovery vs. Patterns vs. Search vs. Recaps
The four analytics surfaces answer four different questions.
Discovery finds signal. Patterns pin it down. Insight Search explains it. Recaps deliver the update.
Limitations
- Discovery quality scales with the volume and diversity of production traffic. Very low volume or homogeneous traffic produces fewer clusters and weaker anomaly detection.
- Discovery surfaces themes across runs. To catch a single high-impact failure, use Patterns or alerts on specific events.
- Recaps are generated on a schedule. Real-time alerting on discovered issues is not yet exposed; use pattern alerts for the events you want in real time.