Keep the lesson for the next incident
Document intended comparability
State which dimensions must match and which differences are deliberate, including population, filters, units, aggregation, and coverage. Explain how a benchmark is selected or revised. Keep these rules connected to the displayed label so a familiar word such as baseline does not conceal several incompatible meanings across reports.
Retain matched and mismatched examples
Keep a small correct comparison and one controlled context mismatch with independent values and expected interpretation. Include a zero or unavailable reference where relevant. Record why selection and arithmetic are separate so future checks do not simplify the example into a correct percentage calculation over whichever values the report happens to return.
Review reference and measure changes
Assign review when baseline selection, shared filters, measure definitions, or saved-report behavior changes. Compare the fixture and visible context across affected surfaces. Track confirmed comparability problems separately from real outcome changes, while preserving the evidence needed to distinguish a shifted reference from a genuine change in the measured behavior.
What to carry forward
Close with a traceable comparison contract, contrasting fixtures, and an owner for baseline or measure changes. Preserve unresolved context explicitly. The next reader should be able to explain what changed and relative to what, without guessing from a percentage or assuming that a benchmark label guarantees a like-for-like reference.
Technical background: Metabase documentation.
Keep the decision with the work.
Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.
Create an account See the product workflow