Aglet

Prioritize Incorrect or Unclear Report Baselines

Baseline defects deserve priority when readers act on a comparison that silently changes the question between its two sides. A missing explanatory label and an incorrectly selected population may need different fixes. Establish the intended comparison and observed mismatch before ranking the issue solely by the size of the displayed percentage change.

Decide where the work belongs

  1. Identify the changed decision meaning

    Compare the current and baseline contexts and describe what readers are likely to infer from the displayed difference. Record demonstrated mismatches in population, measure definition, or coverage. Keep actual decision consequences separate from speculative harm, and distinguish a valid but poorly explained benchmark from a baseline that violates the report’s stated contract.

  2. Scope reused baseline definitions

    Find reports and saved views sharing the same benchmark, filter logic, or comparison expression. Compare a known matched case as a control. Estimate reach from those uses rather than all change indicators, and note whether exported or scheduled versions omit context that the interactive report makes available through a definition or selector.

  3. Assess clarification and correction options

    Check whether readers can inspect both definitions or choose a genuinely comparable reference. Determine whether a label, selection rule, or calculation needs correction. Include the effort of revisiting already shared results when the original baseline context is unavailable or changed after the report was produced and cannot be reconstructed reliably.

What to carry forward

Record priority with the intended comparison, confirmed mismatch, report reach, and available context. Assign a trigger for another shared baseline or silently changed definition. Keep the queue decision focused on a meaningful like-for-like comparison rather than reducing an alarming percentage through a more convenient but equally mismatched reference.

Technical background: Metabase documentation.

Keep the decision with the work.

Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.

Create an account See the product workflow