Build a useful investigation brief
Build the obligation map
Save request, context, tools, conditions, priority, expected state, output, and actual actions. Mark every requirement satisfied, missed, contradicted, or unobservable. Preserve format and exclusion details that a summary might omit.
Locate the first violation
Compare instruction order, dependency, branch condition, tool action, output format, and completion state. Identify whether a later failure followed an earlier miss. Separate an invalid interpretation from an instruction that was genuinely ambiguous.
Replay the boundary
Test with clarified priority, complete inputs, conflicting constraints, and a minimal format. Change one condition at a time. State whether the fault persists and whether the request contract needs an explicit clarification case.
What to carry forward
The investigation brief should link request obligations, actual behavior, first violation, replay, and ambiguity. End with a localized compliance fault or evidence gap. Do not label optional helpful detail as a failure when no instruction prohibited it; preserve the request's stated constraints as the boundary.
Technical background: OpenAI evaluation guidance.
Keep the decision with the work.
Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.
Create an account See the product workflow