Establish what is happening
Define role boundaries
Record each agent’s responsibility, inputs, outputs, authority, completion signal, and allowed handoff. State which shared state is authoritative. Separate coordination quality from the quality of each individual contribution in review.
Map coordination paths
Include serial handoffs, parallel work, conflicting findings, missing messages, delayed responses, duplicate effort, and recovery after a failed agent. Capture shared state, message order, dependencies, and expected owner at each boundary.
Bound the score
Specify whether evaluation measures role selection, message completeness, dependency handling, shared-state consistency, conflict resolution, or outcome. Mark hidden state and unobserved messages as evidence gaps rather than coordination failures in scoring.
What to carry forward
The triage output is a coordination contract with roles, shared state, messages, dependencies, conflict rules, and score limits. Stop when reviewers can see who owns each step. Keep results provisional when a message or state transition is hidden. Record the missing owner and next check before accepting the synthesis.
Technical background: Agent evaluation research on arXiv.
Keep the decision with the work.
Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.
Create an account See the product workflow