Agent decisions
Routing, tool choice, continue or stop: the fast decisions inside an agent.
An agent makes many small decisions between the big ones: which specialist should take this, which tool next, is the task done, should a person step in. Asking a language model to write its way to each of these makes the agent slow and expensive. Hartley returns them as numbers, in one pass, while your code keeps control of what happens next.
Route a request
{
"state": {"message": "My card was charged twice for order 4411", "customer_tier": "pro"},
"questions": {
"route": {
"type": "choice",
"instructions": "Which team should handle this message?",
"criteria": {
"billing": "Charges, refunds, invoices, payment methods",
"orders": "Delivery, returns, order status",
"account": "Login, profile, plan changes",
"human": "Angry, legal, or anything sensitive"
}
},
"urgent": {"type": "noul", "instructions": "Does this need an answer within the hour?"}
},
"options": {"abstain": false}
}a = answers["route"]
team = a["choice"] if a["confidence"] >= 0.5 else "human"Choose the next tool
Your code lists the possible actions; Hartley picks one; your code runs it.
"questions": {
"next_tool": {
"type": "choice",
"instructions": "Given the conversation so far, which tool should the agent call next?",
"criteria": {
"lookup_order": "Find an order by id or email",
"issue_refund": "Refund a payment",
"send_reply": "Answer the user; no tool needed",
"handoff": "Hand the conversation to a person"
}
}
}The answer is always one of your tools: Hartley can't invent a tool name or malformed arguments, because it doesn't write either.
Continue, stop or escalate
At the end of each step:
"questions": {
"done": {"type": "noul", "instructions": "Has the user's request been fully completed?"},
"stuck": {"type": "noul", "instructions": "Is the agent repeating itself or going in circles?"},
"escalate": {"type": "noul", "instructions": "Should a person take over now?"}
}Stop when done is high, hand over when stuck or escalate is, and keep going otherwise.
Keep it fast
- One call per step, all questions in it. They share the read.
- Send the recent context, not everything. The decision usually depends on the last few turns and the tool results; shorter states answer faster.
- Ask again cheaply. Asking about the same state again is much faster than the first read, so a follow-up question in the same step costs little.
- Always have a default. If a decision isn't confident, fall back to the safe branch rather than the most likely one.