> ## Documentation Index
> Fetch the complete documentation index at: https://docs.thecontextcompany.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Measuring AI agent outcomes and success signals

> Attach operational context to production runs and measure resolution, escalation, and failure signals across every conversation.

Agent quality should be measured by whether the agent accomplished what the user needed, not by whether the transcript looked reasonable. The Context Company gives you two ways to measure that: **behavioral signals it detects automatically** from every session, and **operational context you attach** through metadata and feedback for the outcomes you already track elsewhere.

<Note>
  The Context Company does not ingest generic downstream business outcomes such as retention, LTV, CSAT scores, or subscription state. Success signals come from session-level behavior detected in the trace and from any structured metadata you attach at instrumentation time. See [Limitations](#limitations) below.
</Note>

## Two sources of outcome data

<CardGroup cols={2}>
  <Card title="Detected from session behavior" icon="wave-pulse">
    Resolution, escalation, frustration, confusion, and task-failure signals derived from the conversation and execution trace. No configuration required.
  </Card>

  <Card title="Attached by your application" icon="tag">
    Any structured field you send with a run: `resolved`, `escalated`, `workflow_completed`, `plan_tier`, `experiment`, or anything else. Sliced anywhere in the product.
  </Card>
</CardGroup>

## What The Context Company detects automatically

Every conversation is scanned for behavioral signals that indicate whether the interaction went well. These signals feed [Patterns](/analyze/patterns), [failures](/analyze/failures), [topic clusters](/analyze/topics-and-use-cases), and [recaps](/analyze/recaps).

* **Resolution vs. abandonment.** Session-level signals such as repeated asks, contradictions from the user, or an abrupt session ending mark conversations that appear to have gone unresolved even when they never generated an explicit complaint.
* **Escalation.** Patterns of hand-off language and follow-up sessions surface conversations that escalated to a human without needing a ticket-system integration.
* **Frustration and confusion.** Built-in Patterns run on every conversation and flag negative sentiment or user confusion.
* **Task failure.** The built-in task-failure Pattern flags conversations where the user's goal was not met, including cases where the transcript looked successful but the underlying tool calls or retrieval failed.
* **Silent failures.** Tool errors, empty results, and contradictions between what the agent said and what the tool returned are visible in the [trace](/investigate/traces) and drive the [failures](/analyze/failures) view.

These signals are computed from data The Context Company already has. You do not attach them; you segment and filter by them.

## What you can attach through metadata

Anything your application already knows about the outcome or context of a run can be sent as custom metadata and then used to filter or segment every analysis. Common examples:

**Support agents**

* `resolved` (`true` / `false`)
* `escalated` (`true` / `false`)
* `human_edited` (`true` / `false`)
* `reopened` (`true` / `false`)
* `resolution_status` (`resolved`, `pending`, `abandoned`)

**Sales and CRM agents**

* `workflow_completed`
* `meeting_booked`
* `suggestion_accepted`
* `record_updated`

**Product and copilot agents**

* `task_completed`
* `workflow`
* `feature`
* `plan_tier`

Custom metadata is arbitrary key-value data. Use the field names your team already uses; The Context Company does not require a specific schema. See the [Vercel AI SDK](/frameworks/vercel-ai-sdk/index#custom-metadata) or [custom instrumentation](/frameworks/custom-instrumentation/typescript/setup) pages for the syntax in your framework.

<CodeGroup>
  ```ts Vercel AI SDK theme={null}
  await generateText({
    model,
    prompt,
    experimental_telemetry: {
      isEnabled: true,
      metadata: {
        "tcc.runId": crypto.randomUUID(),
        "tcc.userId": user.id,
        "tcc.orgId": org.id,
        // Operational context and outcomes:
        resolved: true,
        escalated: false,
        workflow: "refund-request",
        plan_tier: "enterprise",
      },
    },
  });
  ```

  ```python Python custom instrumentation theme={null}
  with tcc.run(metadata={
      "tcc.userId": user.id,
      "tcc.orgId": org.id,
      "resolved": True,
      "escalated": False,
      "workflow": "refund-request",
      "plan_tier": "enterprise",
  }):
      ...
  ```
</CodeGroup>

### Feedback as an explicit outcome signal

[Feedback](/analyze/feedback) is the one first-class explicit outcome The Context Company records directly: thumbs up or down plus optional free-text, attached to a specific run. If your application already surfaces a rating, an approval step, or a "did this help?" prompt, send the result through the feedback API and it becomes a filter across every view.

## Analyzing outcomes

Anything attached to a run (detected or supplied) can be:

* Filtered on in the runs list, [failures](/analyze/failures), [Patterns](/analyze/patterns), and [topic clusters](/analyze/topics-and-use-cases).
* Grouped by in [Insight Search](/analyze/insight-search) responses.
* Segmented in [recaps](/analyze/recaps).
* Queried through [MCP](/access-data/mcp) and the [REST API](/access-data/api).

## Example questions

* Which recurring failures show up most in escalated conversations?
* Which tools are called most often in sessions that were not resolved?
* What distinguishes resolved conversations from reopened ones?
* Which failure themes affect our enterprise-tier accounts most?
* Did resolution rate improve after we shipped `prompt_version=v14`?
* Which topics have the highest ratio of thumbs-down feedback?

For version comparisons specifically, see [Compare agent versions](/analyze/compare-versions).

## Updating outcomes after a run

Custom metadata is set at ingestion. To attach an outcome that becomes known later (a ticket that gets reopened a day later, a workflow that completes an hour after the agent's final message), submit it through the [feedback API](/analyze/feedback) or by writing a subsequent run tied to the same `tcc.sessionId`.

<Note>
  In-place mutation of custom metadata on a completed run is not currently exposed. If you need to change an outcome retroactively, submit it as feedback or as a follow-up run in the same session, or [contact us](/contact) about your workflow.
</Note>

## Limitations

* The Context Company does not ingest business-system outcomes such as retention, churn, LTV, CSAT survey scores, or subscription state directly. Attach them through custom metadata or feedback if your application already tracks them.
* Detected signals (resolution, escalation, frustration) are inferred from session behavior. They are strong enough for trend analysis and prioritization; use them alongside your own ground-truth data when accuracy on an individual conversation matters.
* Human-review and handoff analysis works today when your application sends the relevant signals as metadata. There is no built-in review workflow inside The Context Company.

## Related

* [Feedback](/analyze/feedback)
* [Compare agent versions](/analyze/compare-versions)
* [Failures](/analyze/failures)
* [Patterns](/analyze/patterns)
* [Users and organizations](/analyze/users-and-organizations)