Tool routing for agents
Pick the next tool in one forward pass instead of a generation step.
The problem
Most agent loops spend a full LLM call deciding which tool to use next. That call is the slowest and least reliable step in the loop: it takes seconds, it occasionally returns a tool that does not exist, and when it is unsure it does not tell you.
What Needle replaces
The planner call that returns a tool name as JSON.
What you do with the answer
- Gate on the probability: run the tool when P(tool) is above your threshold, otherwise fall back to the full model.
- Ask a Noul alongside it: "Should I clarify first?" gives you a cheap escalation signal.
- Score the risk of the action so irreversible tools need a higher bar than read-only ones.
{
"model": "needle-1",
"state": "User: How did last quarter's revenue compare
to forecast? Tools: sql, web_search, calculator.",
"questions": {
"tool": {"type": "choice",
"criteria": {"sql": null, "web_search": null,
"calculator": null}},
"clarify": {"type": "noul",
"instructions": "Ask a clarifying question first?"},
"risk": {"type": "score",
"criteria": ["read only", "writes", "irreversible"]}
}
}200 answers 1 forward pass
tool choice → sql
sql 0.91
web_search 0.06
calculator 0.03
clarify noul → 0.12
P(true) 0.12
risk score → 0.07
read only 0.94
writes 0.05
irreversible 0.01
0 output tokens Illustrative response