Aglet

Triage Retrieval Coverage for Agent Tasks

A grounded answer can still fail because retrieval omitted the passage the task needed. Triage coverage before judging generation. Define required evidence and acceptable sufficiency, then separate retrieval gaps from an agent that ignored retrieved context while tracing the retrieval path.

Establish what is happening

  1. Define evidence needs

    List claims, entities, dates, constraints, relationships, and source types required for the task. State whether one supporting passage is enough or whether coverage must span several sources. Record provenance and version.

  2. Map retrieval slices

    Include complete, partial, redundant, noisy, conflicting, stale, and no-result contexts. Record query, corpus, filters, ranking, source metadata, and expected answerable portions. Mark cases where the agent can answer without retrieval.

  3. Bound the score

    Specify whether coverage measures recall of required evidence, source quality, ranking position, context sufficiency, or downstream answer support. Keep retrieval and generation labels separate. Do not infer coverage from answer fluency.

What to carry forward

The triage output is a retrieval contract with evidence needs, source boundaries, coverage slices, provenance, and score limits. Stop when reviewers can tell what was missing. Keep sufficiency provisional when required claims or source versions are undefined. Record the missing evidence and its owner before scoring an answer.

Technical background: Ragas evaluation documentation.

Keep the decision with the work.

Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.

Create an account See the product workflow