Check whether the outcome improved
Set tradeoff checks
Require matched workload, quality dimensions, outcome floor, latency, tokens, tool calls, retries, review effort, failure cost, sampling, and variance rule. Define excluded costs and decision thresholds for each candidate path.
Review frontier points
Apply checks to easy, difficult, tool-heavy, retrying, long-context, failed, and reviewed tasks. Inspect slice quality beside cost. Record a lower-cost path that misses the floor and a slower path that meets it.
Approve the decision scope
State which workload, quality floor, cost model, and candidate paths the comparison supports. Record owner, reviewer, date, and retest trigger. Mark results inconclusive when a material cost or quality dimension is absent.
What to carry forward
The verification record should show matched tasks, quality floors, cost components, outliers, variance, and scope. Approve only the tradeoff observed. Reopen it when workload, model, tools, review effort, cost accounting, or quality requirements change. Record the limiting dimension and owner before using the comparison to set a default path.
Technical background: Agent evaluation research on arXiv.
Keep the decision with the work.
Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.
Create an account See the product workflow