Aglet

Investigate why a partial rollout diverges by cohort

The investigation should explain what differs between exposed and control cohorts and test whether code, configuration, data, routing, or traffic composition accounts for it. Preserve cohort definitions so another reviewer can reproduce the comparison after exposure changes.

Build a useful investigation brief

  1. Freeze cohort definitions

    Capture version, flag, region, tenant class, device, and time filters used to define each cohort. Check that records remain stable during the analysis. If membership can change, snapshot the query and note its limits.

  2. Compare the execution path

    Trace requests or jobs through routing, dependencies, configuration, and data validation. Identify the first step where outcomes diverge, then compare a matching control at that step instead of comparing only final errors.

  3. Challenge the leading explanation

    Look for counterexamples: exposed successes, unexposed failures, a second region with the same version, or a different data shape. These tests can weaken a rollout hypothesis and prevent a configuration coincidence from becoming a conclusion.

What to carry forward

The investigation brief should include stable cohort queries, a first divergent step, representative records, and competing explanations. Conclude with a confidence level and a bounded follow-up that can be repeated before exposure changes again.

Keep the decision with the work.

Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.

Create an account See the product workflow