> ## Documentation Index
> Fetch the complete documentation index at: https://docs.thecontextcompany.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Tutorial: diagnose a silent tool failure

> A step-by-step walkthrough of how execution-aware analytics catches a bug that transcript-only monitoring would miss.

This tutorial makes the difference between transcript analytics and execution-aware analytics concrete. You will instrument a broken support agent, watch The Context Company surface the problem, and trace it to the exact tool call that failed.

## The scenario

A support agent tells a user their refund has been processed. The transcript looks great:

> **User:** Can you refund my last order?
>
> **Agent:** Done. Your refund has been processed and should appear in your account within 3 business days.

Behind the scenes:

* The agent called `issueRefund` with the user's email instead of their `customerId`.
* The tool returned `{ error: "unknown_customer" }`.
* The model saw the error, decided to reassure the user anyway, and produced the confident response above.

A conversation-only analytics product cannot see this. Nothing in the transcript says "error." The user might not even complain, because the message sounded correct.

## Step 1 — Instrument the agent

Follow your framework's [integration page](/frameworks/vercel-ai-sdk/index) to install and register the SDK. When you call the agent, attach the identity and session metadata:

```ts theme={null}
...tccTelemetry({
  metadata: {
    "tcc.runId": crypto.randomUUID(),
    "tcc.sessionId": conversationId,
    "tcc.conversational": true,
    "tcc.agent": "support-agent",
    "tcc.userId": user.id,
    "tcc.orgId": user.orgId,
  },
})
```

## Step 2 — Send the broken run

Trigger the refund flow. Confirm the run arrives in the [dashboard](https://www.thecontextcompany.com/prod/runs).

## Step 3 — Open the trace

Open the run. You will see:

* **Prompt:** the user's refund request.
* **Model step 1:** the model decides to call `issueRefund`.
* **Tool call `issueRefund`:**
  * **Arguments:** `{ "customerEmail": "user@example.com" }`
  * **Result:** `{ "error": "unknown_customer" }`
  * **Status:** error
* **Model step 2:** the model receives the tool error and generates the final response.
* **Response:** "Done. Your refund has been processed..."

The transcript is fluent. The trace is damning. See [Traces](/investigate/traces) for the full anatomy.

## Step 4 — Confirm the problem is not a one-off

Two ways to find every run with the same shape.

**Failures** ([Failures](/analyze/failures)) groups tool-call errors automatically. Open the failure for `issueRefund` errors and see every run that hit it.

**Insight Search** ([Insight Search](/analyze/insight-search)) can pose the exact question in natural language:

> Find runs where a tool returned an error but the final response said the task was completed. Group by tool and show the top affected users.

You now have the affected runs, the top tools, and the top affected accounts.

## Step 5 — Quantify impact

From the failure or Insight Search result:

* How many runs hit this?
* How many distinct users?
* How many organizations?
* Which agent or workflow is it concentrated in?

Because `tcc.userId` and `tcc.orgId` were attached at ingest, all of this is a filter away. See [Users and organizations](/analyze/users-and-organizations).

## Step 6 — Fix and monitor

Fix the agent (pass `customerId` instead of `customerEmail`; teach the model to surface tool errors instead of ignoring them; add a retry with the correct argument).

Then create a [custom pattern](/analyze/patterns#custom-patterns) to keep an eye on the behavior going forward:

* **Name:** `Ignored tool error`
* **Description:** The response claims the task succeeded even though a prior tool call returned an error.
* **Trigger:** All conversational runs.
* **Alerts:** Slack, threshold-based (for example 3 detections in 60 minutes).

Ship the fix. If the pattern goes quiet, the fix worked. If it fires again, you get a Slack alert with links straight into the offending runs.

## What just happened

* **Discovery** — Failures + Insight Search surfaced runs where the transcript said "success" but a tool returned an error.
* **Root cause** — the trace showed the exact tool, the exact arguments, and the model step that ignored the error.
* **Impact** — the same query answered how many users and organizations were affected.
* **Improvement** — a custom pattern now watches for the behavior across every future run, and Slack raises the alarm on regressions.

This is the loop The Context Company is designed around: capture the execution, discover what is failing, understand why, quantify who is affected, ship a fix, and monitor continuously.

## Related

<CardGroup cols={2}>
  <Card title="Traces" icon="diagram-project" href="/investigate/traces">
    The trace anatomy in detail.
  </Card>

  <Card title="Patterns" icon="magnifying-glass" href="/analyze/patterns">
    Custom classifiers for behaviors like this one.
  </Card>

  <Card title="Insight Search" icon="comment-question" href="/analyze/insight-search">
    Natural-language investigation.
  </Card>

  <Card title="MCP" icon="plug" href="/access-data/mcp">
    Do this workflow from your IDE.
  </Card>
</CardGroup>