Aglet

Investigate relevance explanation gaps with ranked traces

An investigation should show which inputs affect a disputed position and which cannot be observed. Replay a representative query, compare a control, and vary one factor at a time. Keep the user task in view so a technical score is not mistaken for useful relevance.

Build a useful investigation brief

  1. Build the factor trace

    List query, result, matched fields, filters, sort, freshness, boosts, pins, index, and visible explanation. Include a clear control and a disputed row. Record missing factors as unknown rather than inferring them from position.

  2. Vary one ranking input

    Hold the query and result set fixed while changing one boost, field, freshness, or sort input where possible. Compare final order and destination usefulness. Record a non-change because it can rule out an attractive but irrelevant hypothesis.

  3. Write the explanation brief

    State the first factor that changes order, the visible evidence gap, and competing explanations. Recommend one bounded trace, label, or review experiment with a pass condition. Avoid claiming full ranking causality from a small sample.

What to carry forward

The investigation is complete when a disputed position has an evidence trace or a named hidden boundary and a bounded next experiment. Stop when the brief answers the reviewer’s question without overclaiming score causality. Preserve task context and control queries for verification.

Technical background: Elastic documentation.

Keep the decision with the work.

Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.

Create an account See the product workflow