Aglet

Investigate the Cause of Missing Data Fields

Investigation turns a missing-field symptom into a testable explanation. Follow the same records across each boundary, compare successful and failed examples, and inspect the code or mapping that changes the value. Keep competing explanations visible until a controlled comparison rules one in or out.

Build a useful investigation brief

  1. Pair complete and incomplete rows

    Select matched records from the same source and time window, with one retaining the field and one losing it. Compare payload shape, type, encoding, optional flags, and processing version. A paired sample makes a broad claim testable without relying on anecdotal rows.

  2. Reproduce the boundary

    Run the smallest deterministic transformation that includes the suspected stage and capture its input and output. Check parser branches, join keys, field renames, filters, and serialization rules. Record the exact fixture, command, code revision, and observed result for repeatability.

  3. Test alternatives

    Compare a source omission, schema drift, mapping mismatch, and conditional branch as separate hypotheses. Change one relevant input at a time, then compare outputs. If the evidence cannot distinguish explanations, state the missing observation instead of promoting a likely story to cause.

What to carry forward

Produce an evidence-backed brief identifying the first reproducible loss boundary, the compared fixtures, the tested hypotheses, and the remaining uncertainty. A useful investigation ends with a smallest safe change or a named evidence request. It does not claim root cause from correlation alone.

Keep the decision with the work.

Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.

Create an account See the product workflow