Aglet

Verify Confidence Matches Agent Evidence

Confidence verification checks support strength against the exact wording used. Review representative direct, derived, interpretive, contradictory, and incomplete cases, then compare evidence, alternatives, limits, and labels. Verify a transparent ordinal rule or wording standard rather than assuming stronger prose means stronger proof.

Check whether the outcome improved

  1. Define certainty levels

    Write observable criteria for cautious, supported, and strong language using claim type, evidence coverage, comparison, freshness, and unresolved limits. Include a rule for unsupported or conflicting cases. Avoid numerical precision unless the workflow has evidence and governance for that scale.

  2. Exercise matched cases

    Use paired cases with the same question and progressively different evidence: complete, partial, stale, contradictory, and non-discriminating. Compare claims, cited support, alternatives, uncertainty, and confidence language. Confirm the label changes only when the evidence standard or decision boundary changes.

  3. Review decision wording

    Check that a human can tell what is known, inferred, unresolved, and worth verifying from the result. Confirm strong language includes its support and limits, while incomplete cases request a next observation or state a provisional choice. Retain cases and reviewed outputs.

What to carry forward

Accept confidence handling when wording follows explicit support criteria, matched evidence cases change as expected, alternatives and limits remain visible, and incomplete results stay provisional. Keep verification open for ungoverned numerical labels or incomparable examples. Report claim types and evidence cases covered.

Keep the decision with the work.

Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.

Create an account See the product workflow