Aglet

Prioritize Agent Regression Gate Coverage

Regression gates should focus on behavior that matters and changes often enough to need a fast signal. Prioritize protected slices, hard constraints, and thresholds with repeated ambiguity. Keep low-impact metric movement from making the gate noisy for release reviewers and owners.

Decide where the work belongs

  1. Rank gate consequence

    Describe what a missed slice regression or false block could change. Give weight to user-visible behavior, irreversible release choices, safety-critical constraints, and changes with broad task impact in production decisions.

  2. Compare gate coverage

    Review baseline quality, protected slices, aggregate scores, variance, evaluator changes, borderline cases, and infrastructure faults. Identify which gap changes the next action. Prefer a small diagnostic gate over a broad unstable threshold.

  3. Choose a gate queue

    Select dimension, slice, threshold, baseline, owner, and review date. Defer low-impact dimensions with a trigger. If variance is unknown, prioritize calibration or repeat runs before tightening the threshold for release.

What to carry forward

The priority output is a gate queue tied to missed-regression consequence, protected slices, false-block cost, and baseline quality. Start where a threshold can change release action. Keep noisy dimensions separate from decisive gates. Record the protected slice and owner before accepting a pass based on aggregate score without protected slices.

Technical background: OpenAI evaluation guidance.

Keep the decision with the work.

Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.

Create an account See the product workflow