Overview
Alerts watch one number about your runs or spans (latency, cost, error rate, tokens, tool calls) and notify Slack or a webhook the moment it crosses a threshold you set.
An alert watches one number about your agents and tells you when it crosses a line you drew. Latency past eight seconds. Error rate above 5%. Spend over $50 an hour. Trodo recomputes that number every minute over a rolling window, and the first time it breaches, you hear about it in Slack or on a webhook.
Alerts are where every threshold lives: the number, the line and the window are yours. Evals read the meaning of the work, and issues are the problems Lucid files from what alerts and evals catch.
Alerts, issues or evals?
All three answer "watch my agents for me". They are not the same job.
| You get told when | You define | |
|---|---|---|
| Alert | A number you chose crosses a line you drew. | The measure, the threshold, the window. It fires on your terms, whether or not Trodo thinks it is interesting. |
| Issue | Lucid files a problem from failing traces, or new evidence reopens one you closed. | Nothing up front. Track issues sets up the alert, eval and agent that feed it in one click. |
| Eval | A check you wrote fails on a run, a span or a conversation. | The question itself, and what a pass means. This is the one that catches failures with no error in them. |
Use an alert when you already know what "too far" looks like: an SLA, a budget, a latency target your team agreed on. An alert can also watch what an eval decides, which is how self-improving evals are built.
The mental model
Four ideas, in order:
Scope
Run or span. A run is a whole agent execution; a span is one step inside it: an LLM call, a tool call, a retrieval. See Runs and spans.
Measure
The number: duration, cost, error rate, tokens, tool calls, users, satisfaction. Each comes with the aggregations that make sense for it: a count has none, a duration has p95.
Filter
Which rows count. Without one the alert watches everything; with one it watches only checkout-agent, or only spans on gpt-4o.
Condition
The threshold and the window: above 8 s, over the last 15 minutes. The window is rolling, it always ends now.
Put together, an alert reads as one sentence: "p95 duration of spans on charge_card, above 5 s, over the last 15 minutes."
How an alert is evaluated
- Every minuteTrodo reads the rows in your window, the last 15 minutes, the last 24 hours, whatever you set, and keeps only the ones your filter matches.
- Reduce to one numberThose rows become a single value using your aggregation: a count, a rate, an average, a p95.
- Compare, and notify on the crossingIf the value breaches your threshold and it wasn't breaching before, the alert fires and the notification goes out.
The window is rolling, not a fixed bucket. A 15-minute window at 10:04 means 09:49 to 10:04; a minute later it means 09:50 to 10:05. Nothing waits for a clock boundary.
It fires on the crossing, not on the breach
This is the single most important thing to know about alerts, and it is what keeps a channel worth reading.
An alert notifies once, at the moment it goes from healthy to breaching. While it stays breached, nothing more is sent: a latency problem that lasts an hour is one message, not sixty. When the value comes back under your threshold the alert quietly returns to OK; no "recovered" message is sent either. If it breaches again later, that is a new crossing and you hear about it again.
The three states
| State | What it means |
|---|---|
| OK | The number was computed and it is inside your threshold. |
| Alerting | The number is breaching. It notified on the crossing and will not notify again until it recovers first. |
| No data | There was nothing to measure, or not enough. The alert cannot fire from this state, see Not enough data. |
| Paused | You turned the alert off. It is not evaluated at all. |
What alerts do not do
Worth knowing up front, so you don't go looking:
- No grouping. An alert produces one number for the whole window, not one per agent or per tool. To watch three agents separately, make three alerts with a filter each: which also lets you give each one its own threshold, which you usually want anyway.
- No "recovered" notification. Only the crossing into breach is announced.
- No anomaly detection. An alert compares against the fixed number you typed, not against last week. Set the line from what normal looks like: create the alert, leave it a day, and read its chart.
Next
- Create an alert. The form, field by field
- Measures and aggregations. The full catalog of what you can watch
- Filters: narrowing what counts
- Notifications: Slack, webhooks, and the message
- Monitoring alerts: the list, the chart, the history
Fixing an issue
Copy an issue as a prompt for any coding agent, hand it to Cursor or Claude Code to push a fix branch, read it over MCP, then close it, and learn when the fix did not hold.
Overview
Alerts watch one number about your runs or spans (latency, cost, error rate, tokens, tool calls) and notify Slack or a webhook the moment it crosses a threshold you set.