Decide where the work belongs
Rank omission consequence
Describe what a missing claim, constraint, date, or relationship could change. Give weight to tasks where the answer sounds complete despite an absent source and where users cannot easily inspect retrieval.
Compare coverage difficulty
Review direct lookup, multi-source, long-context, noisy, conflicting, stale, and no-result slices. Identify which cases distinguish retrieval failure from generation failure. Prefer slices with explicit evidence requirements and provenance for fair review.
Choose a retrieval queue
Select task, required evidence, corpus slice, owner, and review date. Defer low-impact ordering changes with a trigger. If source metadata is weak, prioritize provenance and coverage measurement before tuning rank.
What to carry forward
The priority output is a coverage queue tied to omission consequence, source difficulty, user reliance, and evidence quality. Start where retrieval can silently invalidate a result. Keep ranking polish separate from the presence of required evidence. Record the missing claim and owner before tuning ranking for the next run.
Technical background: Ragas evaluation documentation.
Keep the decision with the work.
Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.
Create an account See the product workflow