Aglet

Learn from recurring flaky tests in regression suites

Recurring flakiness erodes trust in every nearby result. Review prior episodes for repeated timing, ordering, resource, environment, or assertion patterns. Preserve the context needed to recognize recurrence without treating all intermittent failures as one class.

Keep the lesson for the next incident

  1. Compare failure signatures

    Group episodes by test identity, assertion, seed, ordering, runtime, resource, and product path. Keep distinct signatures separate when they only share intermittent timing.

  2. Record the reliable boundary

    Document the condition that separates stable and unstable runs and the product behavior the test must continue to cover. State what evidence is sufficient to call a future run comparable.

  3. Define a quality recurrence rule

    Create a review trigger for the same failure signature or the same uncontrolled boundary. Name the owner, run context to capture, and decision that should not rely on the unstable result alone.

What to carry forward

The lesson should preserve one reliable run context and a recurrence rule for the observed signature. Keep it conditional until later executions show that the test remains both stable and faithful to the behavior it protects. Record the tested boundary and the condition that made the failure repeatable, then add a future review trigger for harness, fixture, concurrency, and cleanup changes.

Keep the decision with the work.

Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.

Create an account See the product workflow