Detectors

The catalog of failure detectors, how a pattern becomes an issue, and how severity is scored.

Detectors are the rules that find failures. Each one looks for a specific pattern in your runs, spans, or UX events, groups matching evidence, and raises an issue when the pattern recurs enough to matter. They fall into five categories.

Tool failure

Something went wrong calling a tool.

Detector	Catches
Hard failure	5xx and transport errors (timeouts, connection resets).
Validation failure	Bad arguments, 4xx, or schema errors.
Rate limit storm	Bursts of 429 / quota / throttling.
Auth failure	401 / 403, expired or invalid tokens.
Latency outlier	A tool returns ok but far above its usual p95.
Silent empty	A tool returns ok but with no data.
Retry exhaustion	The same tool is retried repeatedly and every attempt fails.

Tool misuse

The agent called tools wrongly even when they didn't error.

Detector	Catches
Wrong tool	A different tool was the right one.
Missing tool	A tool should have been called and wasn't.
Unnecessary tool	A tool call that wasn't needed.
Tool when known	Called a tool when the answer was already known.
Wrong order	Tools called in the wrong sequence.
Stale args	Called with outdated arguments.
Redundant calls	The same call repeated needlessly.
Parallel when serial	Calls run in parallel that depend on each other.

Output quality

The final answer has a problem.

Detector	Catches
Hallucination	Claims not supported by the context.
Contradiction	The answer contradicts itself or the source.
Incompleteness	The answer misses part of the request.
Off topic	The answer drifts from the question.
Wrong format	The output doesn't match the requested format.
Tone mismatch	The tone is off for the context.
Language mismatch	The reply is in the wrong language.
Refusal overreach	The agent refuses something it should handle.
Stale information	The answer uses out-of-date facts.

UX rage

Friction in the product, from the browser SDK.

Detector	Catches
Rage click hotspot	Repeated frustrated clicks on the same element.
Dead zone	Clicks on things that don't respond.
Visible error	Errors the user actually saw.
Slow LCP	Pages that load too slowly.
Form abandon	Forms users start and give up on.
Exit intent burst	Fast bounces off a page.
Scroll dead end	Users scroll to a dead end and leave.

Conversation breakdown

The multi-turn session went wrong.

Detector	Catches
Agent loop	The agent repeats the same tool call with the same args.
Rephrase loop	The user keeps rephrasing the same request.
Silent abandonment	The user drops off mid-conversation.
Tool call storm	An unusually high number of tool calls in one run.
Turn explosion	Far more turns than usual to get somewhere.
Clarification ping-pong	Endless back-and-forth clarifying.
Context amnesia	The agent forgets earlier context.
Escalation handoff failed	The user asks for a human and is ignored.

How a pattern becomes an issue

A detector only raises an issue when a pattern recurs enough to matter. Three thresholds gate it, with sensible defaults:

Threshold	Default	Meaning
Minimum occurrences	10	How many times the pattern must appear.
Minimum affected users	3	How many distinct users it must hit.
Lookback window	15 minutes	The window the counts are measured over.

Matching evidence is grouped by a signature, so the same underlying problem becomes one issue with many members rather than many duplicates.

Severity

Each issue gets a severity score from 0 to 10, combining:

Reach: how many of your users it affects.
Frequency: how often it occurs.
Growth: whether it's trending up.
Business signal: its weight against a business value you care about.
Pattern weight: how serious the pattern is in general.

An issue is also flagged as a regression when its recent rate jumps well above its own baseline, so a resolved problem coming back resurfaces.

Heal to fix an issue
Overview