Establish what is happening
Define answerability
State what context, task information, and confidence support an answer. Identify when the agent should answer, qualify, ask a question, or decline. Keep unsupported certainty separate from a concise answer that omits optional detail.
Map abstention cases
Include complete evidence, missing evidence, conflicting sources, ambiguous intent, out-of-scope request, and safe partial answer cases. Record available context and expected user next action. Mark domain or policy limits that affect interpretation.
Bound the score
Specify whether the evaluation measures abstention choice, explanation, clarification, useful partial help, or calibration. Define over-abstention and under-abstention separately. Do not reward a generic refusal without checking whether it moves the task forward.
What to carry forward
The triage output is an abstention contract with answerability, evidence, response choices, user next action, and score boundaries. Stop when reviewers can distinguish prudent uncertainty from unnecessary refusal. Keep edge cases provisional when request intent is unclear in the current sample.
Technical background: Google DeepMind evaluation research.
Keep the decision with the work.
Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.
Create an account See the product workflow