Keep the lesson for the next incident
Keep the disputed evidence
Record query, result order, matched fields, boosts, freshness, sort, rule context, and user task. State what could not be observed. This keeps a later ranking change from losing the reason the original evidence mattered.
Compare later tuning
Review another query family and a clear control after the explanation change. Look for a factor that still changes order without context or a visible explanation that misleads. Preserve counterexamples and avoid reading improved trust as proof of better relevance.
Refine one review question
Require future ranking reviews to name the user task, show a control query, and record factor provenance or an explicit unknown. Assign ownership and revisit date. Define the observation that would show the practice reduced unsafe tuning changes.
What to carry forward
The learning record links one ranking debate to later evidence and a repeatable explanation check. Stop when another reviewer can apply it to a new query family. Carry forward uncertainty where the ranking system cannot expose every factor or where usefulness remains subjective.
Technical background: Elastic documentation.
Keep the decision with the work.
Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.
Create an account See the product workflow