Aglet

Triage an Adversarial Agent Evaluation Slice

Adversarial cases are useful when they expose a defined weakness rather than merely surprise an agent. Triage the target behavior and valid answer first. Separate stress from ambiguity, and record what a failure proves or leaves explicitly unknown for attribution.

Establish what is happening

  1. Define the target weakness

    State behavior under test, valid task, adversarial condition, allowed context, expected response, and failure severity. Identify whether the case tests distraction, contradiction, boundary pressure, prompt conflict, or another known risk.

  2. Map slice variants

    Include clean control, mild pressure, strong pressure, misleading context, conflicting instruction, missing evidence, and recovery cases. Record item provenance, prompt, evaluator view, and acceptable alternate responses for fair comparison work.

  3. Bound the result

    Specify whether failure indicates susceptibility, evidence gap, task ambiguity, or evaluator limitation. Define slice coverage and false-positive treatment. Do not generalize from one adversarial item to all agent behavior without evidence.

What to carry forward

The triage output is an adversarial slice contract with target weakness, controls, pressure variants, evidence, severity, and scope. Stop when reviewers know what the stressor tests. Keep a result provisional when the clean control or boundary is missing or unclear. Record the control, evidence owner, and review question before labeling susceptibility.

Technical background: Google DeepMind evaluation research.

Keep the decision with the work.

Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.

Create an account See the product workflow