Caching & availability
The SDK caches prompts and serves a stale copy or your fallback if Trodo is unreachable — so a prompt fetch on your hot path degrades rather than takes your app down.
A prompt fetch sits on the hot path of every request your app makes. If that fetch could fail, adding Trodo would make your app less reliable than hardcoding the string — so the SDK is built not to. It follows an availability ladder that degrades instead of failing. For the ladder drawn out, see Concepts.
The availability ladder
In order, on every get:
Fresh cache. Within the TTL (60s by default), the cached prompt is returned immediately — no network call.
Stale — stale-while-revalidate. Past the TTL, the stale copy is returned instantly while the SDK refreshes it in the background. A slow or dead API costs latency on nobody's request.
Fallback. If nothing is cached and the API is unreachable, your supplied fallback prompt is used. This is the cold-start safety net.
Raise. Only when there is no cache and no fallback does the fetch throw.
TTL and the cache selector
get caches under the selector you asked for — a given label and a given version hash cache separately. So production and version: 'a3f9c2' are independent cache entries and never shadow each other.
- TTL default is 60s. Set
cacheTtlSeconds/cache_ttl_secondsto tune how long a fetch stays fresh. cacheTtlSeconds: 0disables caching — every call hits the network. Handy in development so every edit shows up instantly.
Configuring it
The fallback is a minimal prompt of shape { messages, model?, variables? }. Detect that it was used with isFallback / is_fallback and log, so a degraded run is visible.
const prompt = await trodo.prompts.get('refund-agent', {
cacheTtlSeconds: 60, // default 60; 0 disables caching (handy in dev)
fallback: {
messages: [
{ role: 'user', content: [{ type: 'text', text: 'Help with {{question}}' }] },
],
variables: [{ name: 'question' }],
},
});
if (prompt.isFallback) {
logger.warn('running on the fallback prompt — Trodo was unreachable');
}prompt = trodo.get_prompt(
"refund-agent",
cache_ttl_seconds=60, # default 60; 0 disables caching (handy in dev)
fallback={
"messages": [
{"role": "user", "content": [{"type": "text", "text": "Help with {{question}}"}]},
],
"variables": [{"name": "question"}],
},
)
if prompt.is_fallback:
logger.warning("running on the fallback prompt — Trodo was unreachable")Label changes propagate within one TTL
The label is resolved server-side on every fetch and revalidation, and the SDK caches under the selector you asked for. So when you move production to a new version, running apps pick it up within one cache window — without knowing that labels exist. Lower the TTL if you need faster propagation; raise it to cut fetches.
Prefetch at startup and fail the boot if prompts are unavailable. That turns a runtime prompt-service outage into a deploy-time failure — a far better place to find out. Warm the cache for the prompts you use, and exit non-zero if the fetch throws.
Output format
The response_format stored on a version — null, a JSON object mode, or a named JSON schema — passed through unchanged by compile for you to hand to your provider.
Prompt traceability
Every span that used a managed prompt records the exact version by immutable hash — so a trace always shows precisely which prompt ran, even after a deploy label is later moved.