Thresholds
Thresholds decide when a recurring pattern becomes a signal. Set occurrences, distinct users, and a lookback window per pattern — and tune the noise to your taste.
A pattern matching once is not a problem worth your attention. Thresholds are how you tell Trodo what counts as a real signal. You set them per pattern, and Trodo only fires a signal when a pattern clears the bar.
The three knobs
Every pattern has three threshold settings plus an on/off switch:
| Setting | What it does |
|---|---|
| Trigger occurrences | How many times the pattern must occur — for the same entity — before it fires. Filters out one-off blips. |
| Minimum distinct users | How many different users the pattern must hit. Stops a single noisy user (or your own test traffic) from raising a signal. |
| Lookback window | The rolling time window the counts are measured over. Occurrences older than this drop out of the count. |
| Enabled | Turn the pattern on or off entirely. Disabled patterns are never scored and never fire. |
All three counts are evaluated together, over the same rolling window, per entity — a specific tool, LLM node, or page. A pattern fires only when it clears occurrences and distinct users inside the lookback.
A worked example
Take Hard error on your search_orders tool, with:
- Trigger occurrences: 10
- Minimum distinct users: 3
- Lookback window: 15 minutes
Trodo raises a signal the moment search_orders throws at least 10 hard errors, hitting at least 3 distinct users, within any rolling 15-minute window. Nine errors, or ten errors all from one user, stay quiet — that's the noise you asked it to swallow.
Where to set them
Thresholds live in Settings → Issue thresholds. Patterns are grouped by family (Tool failure, Tool misuse, Response quality, Generation quality, UX rage). You can:
- Edit trigger occurrences, minimum users, and lookback inline for any pattern.
- Toggle a single pattern on or off, or flip a whole family at once.
- Reset all thresholds back to Trodo's defaults if you've over-tuned.
Editing thresholds and enabling patterns is a Growth-plan feature. On the free plan you can see the catalog and the defaults, but the controls are locked.
Defaults
Every pattern ships with a sensible default so signals work out of the box — you don't have to configure anything to start. Defaults are tuned per pattern (a hard crash has a lower bar than a slow page), and a common starting point is around 10 occurrences, 3 distinct users, over a 15-minute window. Adjust from there once you see how a pattern behaves on your traffic.
Tuning guidance
Thresholds are a noise dial. Move them to match how much you trust a pattern and how much it hurts when it's real.
- Too many low-value signals? Raise trigger occurrences or the distinct-user bar, or shorten the lookback so only a genuine spike clears it.
- Missing real problems? Lower the bar. For high-severity patterns (auth failures, hard errors on a critical tool), a handful of occurrences across two users may already be worth a signal.
- Bursty vs. slow-burn. A short lookback catches sharp spikes (a rate-limit storm) but can miss a slow bleed. Widen it for patterns that drip over hours.
- Low-traffic entities. If a tool or node sees little traffic, high user or occurrence bars may never trip — lower them so real failures still surface.
- Silence a pattern you don't care about. Disable it outright rather than setting an impossibly high bar; disabled patterns cost nothing and never fire.
After a signal fires
Once a pattern clears its threshold and fires, that signal enters a short cooldown before it can fire again for the same entity — so a still-broken tool doesn't page you every few seconds. New, distinct problems still fire immediately; it's only the same signal re-firing that's rate-limited. This is automatic and needs no configuration.
Next
Patterns
The catalog of failure patterns behind signals, grouped into five families, with what each one means and why it matters.
AI Score
Seven generation-quality scores grade every LLM node output — grounding, rule adherence, contradiction, trajectory, factual retention, echo, and refusal. What each means, its range, and how to improve it.