Keep the lesson for the next incident
Save the trajectory case
Store task fixture, ordered states, observations, calls, returns, retries, side effects, final answer, score, and interpretation. Include a successful outcome with a faulty intermediate transition and a failure caused by missing evidence.
Improve trace practice
Add state snapshots, transition labels, alternate-path rules, recovery checks, and missing-evidence statuses to evaluation plans. Explain how the change addresses the first divergence rather than merely adding more trace volume.
Watch for path blind spots
Set a signal such as final answers passing while state checks fail, repeated retries without snapshots, or loops counted as success. Assign an owner to sample trajectories and revisit the check when the signal appears.
What to carry forward
The learning record should link the first divergent state to the revised trace practice and recurrence signal. State which transition evidence became required. Keep the lesson tied to the task contract and observed states instead of treating path length as quality over time.
Technical background: Agent evaluation research on arXiv.
Keep the decision with the work.
Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.
Create an account See the product workflow