Aglet

How to learn from job retries and idempotency boundaries

Job-retry learning records what can be repeated, what must happen once, and how a failed job becomes reviewable. Keep the attempt timeline, side-effect boundary, retry classification, and final state contract available for future job or dependency changes and operational handoffs.

Keep the lesson for the next incident

  1. Document the logical job contract

    Describe inputs, stable identity, local and external effects, commit boundary, acknowledgment, retryable failures, permanent failures, and exhausted outcome. Explain how final state is recognized and which values must not duplicate. Keep examples structural and free of sensitive payloads.

  2. Keep a failure-phase replay journey

    Preserve checks for failure before effect, failure after effect, transient recovery, permanent rejection, exhaustion, and safe replay. Define expected attempt history and final records. Assign an owner to update the journey when a job gains a new side effect or dependency.

  3. Recheck after worker-path changes

    Review the cases after transaction, acknowledgment, retry policy, concurrency, dependency, or payload changes. Compare idempotency and final state again. Add a new failure phase when the job's effect boundary moves instead of assuming the old pre-effect and post-effect cases still apply.

What to carry forward

The learning record should retain job identity, attempt and effect boundaries, retry classes, exhaustion contract, replay checks, and recurrence triggers. It is complete when future engineers can tell safe recovery from repeated side effects and can verify the logical outcome independently of attempt count.

Keep the decision with the work.

Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.

Create an account See the product workflow