Caching & availability

The SDK caches prompts and serves a stale copy or your fallback if Trodo is unreachable, so a prompt fetch on your hot path degrades rather than takes your app down.

A prompt fetch sits on the hot path of every request your app makes. If that fetch could fail, adding Trodo would make your app less reliable than hardcoding the string, so the SDK is built not to. It follows an availability ladder that degrades instead of failing. For the ladder drawn out, see Concepts.

The availability ladder

In order, on every get:

Fresh cache. Within the TTL (60s by default), the cached prompt is returned immediately, no network call.

Stale: stale-while-revalidate. Past the TTL, the stale copy is returned instantly while the SDK refreshes it in the background. A slow or dead API costs latency on nobody's request.

Fallback. If nothing is cached and the API is unreachable, your supplied fallback prompt is used. This is the cold-start safety net.

Raise. Only when there is no cache and no fallback does the fetch throw.

Config errors skip the ladder

The ladder exists for availability failures: Trodo being unreachable. A selector that names nothing is a different thing entirely: asking for version: 999 or label: 'prod-ue' on a prompt that exists is a mistake in your code, and quietly handing you the fallback would hide the typo for as long as it ships.

So those throw immediately, even with a fallback configured, with an error that names the actual problem:

version_not_found: the version number or hash doesn't exist on this prompt
label_not_found: no version carries that label
no_versions: the prompt exists but nothing was ever saved to it

Read the code off the error (err.code in Node, e.code in Python) if you want to branch on it. A prompt that is genuinely missing (prompt_not_found) still walks the ladder: deleting a prompt mid-deploy is exactly the outage-shaped event the fallback exists for.

TTL and the cache selector

get caches under the selector you asked for: a given label and a given version hash cache separately. So production and version: 'a3f9c2' are independent cache entries and never shadow each other.

TTL default is 60s. Set cacheTtlSeconds / cache_ttl_seconds to tune how long a fetch stays fresh.
cacheTtlSeconds: 0 disables caching: every call hits the network. Handy in development so every edit shows up instantly.

Configuring it

The fallback is a minimal prompt of shape { messages, model?, variables? }. Detect that it was used with isFallback / is_fallback and log, so a degraded run is visible.

const prompt = await trodo.prompts.get('refund-agent', {
  cacheTtlSeconds: 60,       // default 60; 0 disables caching (handy in dev)
  fallback: {
    messages: [
      { role: 'user', content: [{ type: 'text', text: 'Help with {{question}}' }] },
    ],
    variables: [{ name: 'question' }],
  },
});

if (prompt.isFallback) {
  logger.warn('running on the fallback prompt: Trodo was unreachable');
}

prompt = trodo.get_prompt(
    "refund-agent",
    cache_ttl_seconds=60,   # default 60; 0 disables caching (handy in dev)
    fallback={
        "messages": [
            {"role": "user", "content": [{"type": "text", "text": "Help with {{question}}"}]},
        ],
        "variables": [{"name": "question"}],
    },
)

if prompt.is_fallback:
    logger.warning("running on the fallback prompt: Trodo was unreachable")

Label changes propagate within one TTL

The label is resolved server-side on every fetch and revalidation, and the SDK caches under the selector you asked for. So when you move production to a new version, running apps pick it up within one cache window, without knowing that labels exist. Lower the TTL if you need faster propagation; raise it to cut fetches.