Playground

The Playground is where you iterate on a single case. You give it an input, run it across one or more models at once, and get scored, side-by-side outputs so you can see — concretely — which model and which wording works best. It's the scratchpad; Experiments is the batch test suite that runs a prompt across a whole dataset, and Prompt Management is where a tuned prompt goes to be versioned and shipped.

Three ways to start a run

Pick an input mode at the top of the editor. All three end at the same place — a scored, side-by-side comparison — they just differ in where the input comes from.

Manual mode

A manual run is one conversation — a vertical list of role-tagged messages that go to the model exactly as you write them. Each message has a role:

• System — the rules, persona, and task the model must follow.
• User — the end user's request.
• Assistant — a prior model turn, when you're testing how the model continues a conversation.

Add as many messages as you need in any order. There's no templating and no variables in manual mode — the input is sent as-is. Structured inputs, {{variables}}, and datasets live in Prompt Management and Datasets; use From prompt when you want those.

Scoring reads the roles: your system messages form the instruction being judged, the last user message is the query relevance is anchored to, and the rest is context (see Scores). If there's no user message, scoring falls back to anchoring on the system instruction ("did it do the job?").

From span

Click Import from span and paste the ID of a real LLM span from your traces. Trodo parses the whole recorded input — every turn, in order — into the editor as one conversation, each turn mapped to the role the model accepts (system → system, assistant → assistant, everything else → user). You reproduce the production call exactly, then change one thing and re-run to see the effect. Once imported, every message is fully editable — it behaves like a manual run seeded from real data.

This is the fastest way to answer "why did this specific production call go wrong?" — pull the exact input into the bench and iterate on it.

From a prompt

This mode runs a managed prompt the way your application does — so you're testing the real artifact, not a copy you pasted. It's how you validate a prompt version before you move a deploy label to it.

Open From prompt and:

1. Pick a prompt, then pick a version (v1, v2, …). You're testing that exact immutable version — the same one your SDK would resolve.
2. Set variables — type a value for each {{variable}} in the template, for a one-off run.
3. Mark which variables are the user query. Tag the variable(s) that represent the end user's request (e.g. question, user_message). This drives a faithful split between generating the output and scoring it:
   • Generation — the model sees the compiled template with the non-query variables filled inline where the author placed them, and the query values delivered as the user turn — exactly as production would send it.
   • Scoring — relevance is anchored to the query variables, groundedness to the remaining data variables, and adherence to the compiled system instruction. If you mark no query variable, the whole compiled prompt is sent as a single user turn and scoring falls back to anchoring on the instruction ("did it do the job?").

A live prompt preview shows the compiled template (using your typed values) so you can confirm it renders the way you expect before running.

To run this prompt across an entire dataset instead of one set of values, use Run over a dataset.

Picking a specific version here is the point: run v2 against the same dataset you ran v1 on, compare the scores, and only then move the production label. See Prompt Management for how versions and labels work, and Experiments for running every row at once.

Models

Add one or more models — each is a linked provider, a model, and a temperature. Runs execute in parallel, and the results are ranked by their composite Alignment Score (ties broken by cost, then latency). Add several to compare, e.g. gpt-4o vs claude-sonnet-4-6 vs a cheaper model on the same input.

Scores

Every output is scored on these dimensions:

With 2+ models you also get cross-model divergence and an outlier flag, so you can tell "all models agree" from "this input is model-sensitive."

Relevance, correctness, groundedness, and adherence are deliberately distinct: an output can be on-topic (relevance) but wrong (correctness), or right but unsupported by the data you gave it (groundedness), or perfectly formatted (adherence) but about the wrong thing (relevance).

Each card also has a What to improve section — concrete issues the judge flagged — and, on re-runs, the score delta versus the previous run.

Run history

Every run you make in a playground is recorded automatically — the playground has no separate "versions" to manage; the run history is the record. Open the Runs menu to see recent runs and restore any of them — input, model config, and results — back into the editor. It's an undo for experimentation: try an aggressive rewrite, and if it scores worse, jump back to the run that worked. Reopening a playground loads its most recent run, so you pick up where you left off. (The most recent runs are kept per playground.)

Run over a dataset

The playground runs one case at a time. When you're ready to test across all your cases, click Run over a dataset (in From prompt mode) — it opens the New experiment dialog prefilled with this prompt, so you just pick a dataset and models. The batch run, per-row scores, aggregate, and comparison all live in Experiments.

The division of labour: spot-check and tune a single input here; measure across the whole set — reproducibly and comparably — as an experiment.

Saving your work

• Save keeps the playground setup (its conversation and models) so you can come back to it. Prompt-mode runs don't need a manual save — they're already captured under Runs.
• Save as prompt promotes the current conversation into the prompt registry as a new (or newly versioned) managed prompt: the system messages become the instruction, the last user message becomes the query, and any {{tokens}} you typed are auto-declared as variables. It's the bridge from scratchpad to shippable artifact — available in Manual and From span modes.

Walkthrough

Tuning a new prompt (manual):

1. Open Playground and create one (or open an existing playground).
2. In Manual mode, write your conversation — a System message with the rules, and a User message with the request. Add more messages if you're testing a multi-turn exchange.
3. Add one or more models and click Run.
4. Read the scored cards; expand Dimensions and What to improve. Tweak the wording or swap a model and run again — each re-run shows the score delta.
5. Save as prompt to promote the conversation into the prompt registry as a versioned prompt.

Validating a prompt version before you ship it (from prompt):

1. Switch to From prompt, pick the prompt and the version you're considering. Set variables and spot-check a case or two.
2. Click Run over a dataset to open the New experiment dialog with the prompt prefilled; pick a dataset and your candidate models.
3. Run it, then Compare its aggregate against the current production version's experiment (Experiments → Comparing).
4. If it wins, move the production label to the new version in Prompt Management — no redeploy needed.

Next

• Prompt Management — version a tuned prompt and ship it by moving a label
• Experiments — run a prompt version across many cases at once and compare
• Datasets — build and curate the question set experiments run on
• Curate from production — promote a trace span into a dataset row
• Import from a span — seed the playground from a real production trace