Aglet

How to investigate intermittent errors with matched traces

An intermittent-error investigation should explain what differs between a failing attempt and a successful one without declaring a cause from timing alone. Trace one matched pair across boundaries, reproduce the smallest pattern in a controlled setting, and test a focused change.

Build a useful investigation brief

  1. Align failing and successful traces

    Choose the same route, method, input structure, account class, release, and nearby time window where possible. Compare admission, validation, queries, locks, dependency responses, serialization, status, and visible state. Record missing telemetry and sampling gaps so silence is not treated as a healthy interval.

  2. Reduce the pattern to one variable

    Hold route and client action fixed while varying one suspect dimension such as payload field, data size, timing, concurrency, dependency response, or release. Repeat both outcomes in a controlled environment. Preserve structural values and redact content so another engineer can reproduce the distinction.

  3. Test one focused reliability change

    Evaluate one intervention such as clearer validation, bounded dependency handling, transaction correction, safe retry ownership, or better error classification. Replay the failing and successful shapes. Check state before and after, response semantics, duplicate effects, and recovery so a lower error count does not hide corruption.

What to carry forward

Deliver an evidence-backed brief with matched traces, reduced variable, first divergent boundary, tested change, and unresolved alternatives. Investigation is complete when implementation has one testable reliability target and verification can replay both failure and success paths with state evidence.

Keep the decision with the work.

Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.

Create an account See the product workflow