Aglet

Verify Cohort Retention Report Cells

Retention verification checks whether each cell tells the same cohort story as its underlying member set. Exercise an entry cohort with early, late, absent, and repeated activity. Compare counts, rates, period labels, and export context, and make cumulative or first-return behavior explicit rather than assumed.

Check whether the outcome improved

  1. Define cell assertions

    Write expected cohort members, denominator, return event, window, retained count, percentage, and label for each fixture. State treatment of re-entry, missing dates, and ineligible members. A pass requires the cell definition to be discoverable from the report or its nearby explanation.

  2. Exercise member timelines

    Run early return, late return, no return, re-entry, and boundary-date cases through table, chart, card, filter, and export views. Compare member sets and values. Inspect whether cumulative curves and period columns reflect the documented rule rather than a convenient visual shape.

  3. Review reader interpretation

    Ask a reviewer to explain who entered the cohort, who was eligible, what each period means, and whether the value is cumulative. Preserve timeline fixture, filter state, report revision, and evidence. Record a bounded result where a surface omits member-level context but keeps correct cells.

What to carry forward

Verification passes when cohort cells, denominators, return windows, and labels agree across reviewed surfaces. Report a bounded pass for uncovered cohort policies. Keep verification inconclusive if late activity shifts columns silently or a cumulative value is presented as a period rate.

Technical background: Metabase documentation.

Keep the decision with the work.

Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.

Create an account See the product workflow