Decide where the work belongs
Rank transition consequence
Describe what an invalid call, stale observation, loop, or skipped precondition could leave behind. Give weight to changes that users cannot easily undo and to failures a polished final answer would conceal.
Compare path difficulty
Review direct, chained, branching, retry, partial, and recovery trajectories. Identify which cases distinguish valid alternatives from redundant work. Prefer paths with observable state boundaries and a clear diagnostic question for trace reviewers.
Choose the trace queue
Select cases, required fields, reviewer, owner, and retest date. Defer low-consequence timing or token variations with a trigger. If state snapshots are unavailable, prioritize evidence capture before ranking more trajectories.
What to carry forward
The priority output is a trajectory queue tied to transition consequence, recovery difficulty, path ambiguity, and trace completeness. Start where an invalid path could survive final-answer review. Keep efficiency analysis separate until the correctness trace is trustworthy before expanding coverage, changing the score, or revising release criteria.
Technical background: Agent evaluation research on arXiv.
Keep the decision with the work.
Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.
Create an account See the product workflow