Aglet

Triage relevance explanation gaps before changing ranking

Ranking debates become expensive when no one can explain why a result appears first. Triage the explanation gap before changing scores. Capture query, matched fields, visible labels, rule inputs, sort, and result task so an opaque order is distinguished from a merely surprising one.

Establish what is happening

  1. Capture the ranked sample

    Record query, filters, sort, result positions, matched fields, freshness, boosts, pins, and useful destination. Include one result whose position is clear and one disputed row. Keep rule or index references alongside the sample.

  2. Separate evidence from inference

    Write which visible or recorded signal supports the order and which explanation is only suspected. Compare textual match, business rule, freshness, and selected sort. A missing explanation is not proof the ranking mechanism is wrong.

  3. Set an explanation boundary

    State which ranking factors can be shown, which need an internal trace, and what a reviewer must know to make a safe change. Preserve the disputed query. Stop when another person can reproduce the question without repeating the entire debate.

What to carry forward

The triage output is an explanation boundary with ranked sample, visible signals, hidden inputs, and task context. Stop when surprise and evidence absence are distinct. If the order is intentional but opaque, route explainability work separately from relevance correction.

Technical background: Elastic documentation.

Keep the decision with the work.

Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.

Create an account See the product workflow