Check whether the outcome improved
Freeze the success rule
Write the primary outcome, acceptable guardrail range, minimum observation window, and result that would count as inconclusive. Confirm that the rule was set before the latest result and that proxy movement cannot substitute for the intended user outcome.
Review matched cohorts
Compare treatment and comparison records using the documented assignment and exclusion rules. Inspect exposure quality, missing events, task completion, and user context. Report meaningful exceptions and any segment whose result moves opposite to the aggregate.
Decide within scope
State which hypothesis clause the evidence supports, weakens, or leaves open. Record the next action and a review trigger. Do not generalize beyond the measured audience, task, instrument, or time window.
What to carry forward
The verification record should show the frozen rule, cohorts, primary result, guardrails, exceptions, and tested boundary. Choose a next action only within that boundary. Mark the result inconclusive when assignment, exposure, or duration prevents a fair comparison.
Technical background: IDEO design thinking guidance.
Keep the decision with the work.
Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.
Create an account See the product workflow