Documentation Index
Fetch the complete documentation index at: https://docs.trodo.ai/docs/llms.txt
Use this file to discover all available pages before exploring further.
Thin wrappers around withSpan. Declare the span once at the function definition site instead of inlining withSpan(...) at every call site.
const fetchUser = trodo.tool('fetch_user', async (id) => db.users.findById(id));
const searchKb = trodo.retrieval('kb.search', async (q) => kb.search(q, { k: 8 }));
const prepareCtx = trodo.trace('prepare_ctx', async (u, t) => ({ history: await loadHistory(u), topic: t }));
const answer = trodo.llm('answer', async (messages) => {
return await client.chat.completions.create({ model: 'gpt-4o-mini', messages, temperature: 0.2 });
}, { model: 'gpt-4o-mini', provider: 'openai', temperature: 0.2 });
@trodo.tool("fetch_user")
def fetch_user(id): return db.users.find(id)
@trodo.retrieval("kb.search")
def search_kb(q): return kb.search(q, k=8)
@trodo.trace("prepare_ctx")
def prepare_ctx(u, t): return {"history": load_history(u), "topic": t}
@trodo.llm("answer", model="gpt-4o-mini", provider="openai", temperature=0.2)
def answer(messages): return client.chat.completions.create(model="gpt-4o-mini", messages=messages)
args as span input, return value as span output, and records thrown exceptions as status='error'. Don’t use llm for frameworks we already auto-instrument (OpenAI, Anthropic, etc.) — you’ll get duplicate spans.
llm auto-extracts tokens from three response shapes: OpenAI (usage.prompt_tokens), Anthropic (usage.input_tokens), Google (usageMetadata.promptTokenCount). For anything else, pass your own:
const generate = trodo.llm('cohere.generate', callCohere, {
model: 'command-r',
provider: 'cohere',
extractUsage: (r) => ({
inputTokens: r?.meta?.tokens?.input_tokens,
outputTokens: r?.meta?.tokens?.output_tokens,
}),
});
trackLlmCall — imperative LLM span
Use when you’ve already made the LLM call and just want to record it, e.g. raw fetch to Ollama / vLLM / a self-hosted endpoint.
const resp = await fetch('http://ollama.internal/api/chat', {
method: 'POST',
body: JSON.stringify({ model: 'llama3.1:70b', messages }),
}).then((r) => r.json());
await trodo.trackLlmCall({
model: resp.model,
provider: 'ollama',
inputTokens: resp.prompt_eval_count,
outputTokens: resp.eval_count,
prompt: messages,
completion: resp,
metadata: { endpoint: '/api/chat' },
});
trackLlmCall is a no-op outside wrapAgent. Pass cost explicitly to override the internal pricing table (negotiated rates, self-hosted zero-cost, etc.).
Feedback
Record a user’s reaction to a run — thumbs / rating / comment. One of satisfaction, rating, comment must be set.
const { result, runId } = await trodo.wrapAgent('support-bot', fn);
// … later, after the user clicks thumbs-up:
await trodo.feedback(runId, { satisfaction: 'positive', rating: 5, comment: 'looks right' });
| Field | Type | Purpose |
|---|
satisfaction | 'positive' | 'negative' | Thumbs-up / thumbs-down column + cluster satisfaction rollup |
rating | number | Any numeric scale (1–5, NPS, eval score) |
comment | string | Free-text note, searchable in the dashboard |
metadata | object | Free-form JSON, e.g. { source: 'thumbs_widget' } |
Custom attributes
Attach arbitrary key/value pairs to any span. Visible in the span detail drawer, searchable in the dashboard.
await trodo.withSpan({ kind: 'tool', name: 'billing.lookup' }, async (span) => {
span.setAttribute('customer_tier', 'enterprise');
span.setAttribute('region', 'eu-west-1');
span.setAttribute('request_id', req.id);
return await billing.lookup(req.customerId);
});
run.setMetadata({ ... }).
Errors
Two levels — they’re not mutually exclusive.
| Level | Trigger | Sets |
|---|
| Span error | withSpan / tool / llm callback throws | status='error', error_type, error_message on the span; increments run’s error_count |
| Run error | wrapAgent callback throws (or run.setErrorSummary(...)) | status='error', error_summary on the run |
withSpan that bubbles all the way up sets both. wrapAgent always flushes on throw — you don’t need try/catch around it just to “rescue” metrics.
For providers that return HTTP 200 with an error body, either throw after checking (makes the span an error), or setAttribute('llm_error', ...) and keep status='ok' (shows up in filters without inflating error_count).
Tokens and cost
Tokens come from the provider’s response; cost is computed at ingest time from the internal pricing table.
| Framework | Token source | Auto-extracted? |
|---|
| OpenAI / Azure OpenAI | response.usage.{prompt_tokens, completion_tokens} | yes |
| Anthropic | response.usage.{input_tokens, output_tokens} | yes |
| Google Gemini / Vertex | response.usageMetadata.{promptTokenCount, candidatesTokenCount} | yes |
| Bedrock Converse | response.usage.{inputTokens, outputTokens} | yes |
| Cohere | response.meta.tokens.{input_tokens, output_tokens} | yes |
| Mistral | response.usage.{prompt_tokens, completion_tokens} | yes |
| Vercel AI SDK | Accumulated across calls (requires experimental_telemetry) | yes |
| LangChain / LlamaIndex | From the underlying LLM wrapper | yes if wrapper supports it |
| Raw HTTP | — | use trackLlmCall |
(input_tokens × input_rate) + (output_tokens × output_rate) from the pricing table. Pass cost explicitly on setLlm or trackLlmCall to override — enterprise rates, cached tokens, self-hosted zero-cost.
Rollups
| Run field | Source |
|---|
total_tokens_in | SUM(span.input_tokens) across kind='llm' spans |
total_tokens_out | SUM(span.output_tokens) across kind='llm' spans |
total_cost | SUM(span.cost) across kind='llm' spans |
span_count | COUNT(span) |
tool_count | COUNT(span WHERE kind='tool') |
error_count | COUNT(span WHERE status='error') |
parentRunId does not cascade tokens from child runs. Use joinRun for unified rollups.
