Choice one of N
Pick one option from up to 255, with a probability for each. Route a ticket, select a tool, pick the next action.
Needle · private preview
Stop asking an LLM to pick an option
Needle is a decision model behind one API call. Send state and typed questions, get back probabilities over the options you declared. About 40 ms, no output tokens, nothing to parse.
{
"model": "needle-1",
"state": "User: How did last quarter's revenue compare
to forecast? Tools: sql, web_search, calculator.",
"questions": {
"tool": {"type": "choice",
"criteria": {"sql": null, "web_search": null,
"calculator": null}},
"clarify": {"type": "noul",
"instructions": "Ask a clarifying question first?"},
"risk": {"type": "score",
"criteria": ["read only", "writes", "irreversible"]}
}
}200 answers 1 forward pass
tool choice → sql
sql 0.91
web_search 0.06
calculator 0.03
clarify noul → 0.12
P(true) 0.12
risk score → 0.07
read only 0.94
writes 0.05
irreversible 0.01
0 output tokens Illustrative response
- median latency, measured
- 41 ms
- output tokens, ever
- 0
- blind-test accuracy, vs 85.8%
- 89.7%
- per million input tokens
- $0.20
Three primitives
Ask typed questions. Get typed answers.
Every question declares its own answer space, so the model can only return an option you defined. Nothing to parse, nothing to retry.
Score ordered rubric
A fractional rating on a rubric of 2 to 10 levels, weighted by probability. Grade severity, rank candidates, rate quality.
Noul P(true)
The probability that a statement holds. Gate an action, flag a record, and set the threshold yourself.
Why a forward pass
Same decision. One of these is an LLM call.
If a step in your pipeline asks a language model to choose, rate or approve something, it is paying for generation to get a label. Needle is built for exactly that step.
| LLM with structured output | Needle | |
|---|---|---|
| Latency | Seconds. Every output token is a decode step. | 41 ms median, 63 ms p95. One forward pass. |
| Cost | Input tokens plus output tokens, on a large model. | $0.20 per million input tokens. Output is $0. |
| Output | Text you parse, validate and sometimes retry. | A probability for each option you declared. |
| Determinism | Sampled. The same input can answer differently. | No sampling. The distribution is the answer. |
| Confidence | Ask for it and get a number the model made up. | Read it off the distribution and set your own threshold. |
Pricing
$0.20 per million input tokens
Output is always free, because there is none. 10M tokens a month on the free tier.
Research
We build agents that learn to plan
The planning lives in the model itself: a single forward pass, self-supervised, learned from observation rather than demonstration. No search at test time. No expert to imitate.
Put a decision model in the loop
Needle is in private preview. Read the docs, then join the list for a key.