Aglet

Verify Consistency of Agent Recommendations

Consistency verification tests whether comparable inputs produce comparable recommendations and whether intentional differences are explainable. Use a controlled case set with shared and varying conditions, then compare decision, evidence, and uncertainty. Verify the contract around disagreement instead of requiring identical prose.

Check whether the outcome improved

  1. Set comparable cases

    Create fixtures with the same question, source artifacts, time window, constraints, and objective, plus paired cases that intentionally vary one condition. Define expected agreement or allowed divergence before review. Include a missing-artifact case with a documented provisional outcome.

  2. Compare recommendations

    Evaluate action, rationale, cited evidence, assumptions, and confidence for each case. Check whether a changed recommendation follows the changed input or criterion. Inspect shared cases for unexplained action, scope, or evidence differences even when their wording appears similar.

  3. Review disagreement handling

    Confirm conflicting valid options identify the policy choice, owner, and next decision rather than silently selecting one. Verify mismatched inputs are labeled and incomparable cases are excluded. Retain case definitions, outputs, revisions, and reviewer notes for repeatable verification.

What to carry forward

Accept the workflow when matched cases agree within the decision contract, changed inputs produce explained differences, and valid alternatives expose their policy choice and uncertainty. Keep verification open for untraceable splits or asymmetric context. Report the cases, revisions, and disagreement classes covered.

Keep the decision with the work.

Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.

Create an account See the product workflow