Aglet

Triage a Product Experiment Before Launch

A product experiment is a learning instrument, not a smaller launch plan. Triage the claim it is meant to test, the user behavior that would count as evidence, and the harms or confounds that could make a result misleading. Make the experiment small enough to stop deliberately.

Establish what is happening

  1. State the hypothesis

    Write who is expected to do what, in which context, and why the treatment could change the outcome. Separate a behavior prediction from a business interpretation. Name the decision that depends on the result and the evidence it cannot provide.

  2. Map the comparison

    Describe treatment exposure, comparison experience, eligibility, exclusions, and observation window. Check that both groups encounter comparable tasks and that assignment does not silently select the most motivated users. Record the eligibility decision for every participant.

  3. List guardrails

    Record failure modes involving access, support burden, data quality, trust, or downstream operations. Define a stop condition and an owner who can review it. Mark any measure that is a proxy rather than the user outcome of interest.

What to carry forward

The triage brief should name the hypothesis, comparison, eligible population, guardrails, and decision boundary. Stop when the team can explain what a result would change and what it cannot show. Do not proceed with an experiment whose primary outcome is undefined.

Technical background: IDEO design thinking guidance.

Keep the decision with the work.

Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.

Create an account See the product workflow