Aglet

Verify relevance explanations on representative queries

Verification should prove that a reviewer can understand the main reason for a result’s position within the promised scope. Test clear, disputed, and out-of-scope queries. Check explanation copy against actual order and keep the user task separate from internal scoring detail.

Check whether the outcome improved

  1. Check a clear result

    Run a query with an obvious field match and confirm the visible explanation or result context supports its position. Record query, sort, and rule state. A clear control prevents a generic factor label from passing without evidence.

  2. Check a disputed result

    Replay the query that motivated the change and compare position, factor summary, and destination usefulness. Test one neighboring query to catch a broad explanation. Record what remains unknown rather than filling the gap with certainty.

  3. Review scope and copy

    Inspect labels, sort, responsive layout, and direct links. Ensure the closure record says which factors were observed and which were not. Leave a partial pass when the explanation helps one query family but cannot support a global ranking claim.

What to carry forward

Verification passes when representative queries have explanations that match observed ordering within stated scope. If hidden factors or one query family remain untested, document them. A plausible label is not proof unless it corresponds to the evidence and user task.

Technical background: Elastic documentation.

Keep the decision with the work.

Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.

Create an account See the product workflow