Aglet

Prioritize shutdown interruption by lost-work and replay risk

Shutdown handling is urgent when a worker can disappear after claiming work without leaving an owner or safe replay boundary. A clean drain with a durable checkpoint can wait behind a mid-step exit that repeats an irreversible effect. Prioritize ambiguity at the last durable boundary and the consequence of recovery.

Decide where the work belongs

  1. Rank the recovery failure

    For each interrupted job, record whether work is lost, replayed, delayed, or safely resumed. Put unowned claims and duplicate-prone effects ahead of a visible interrupted label. Include dependents waiting on the job’s terminal state.

  2. Compare shutdown modes

    Group evidence from graceful drain, forced stop, restart, and worker replacement. Note which modes preserve checkpoints and which leave the claim active. A failure only under one shutdown mode needs a narrower priority than a shared ownership gap.

  3. Choose a recovery-safe action

    Select immediate reconciliation, a drain or checkpoint review, or one phase-controlled shutdown trace. State the readback required before requeue. Do not requeue every active job while its last effect and checkpoint are unknown.

What to carry forward

Queue the highest confirmed loss, duplicate, or ownerless state with shutdown mode and checkpoint evidence. Escalate when restart can repeat effects or leave a claim permanently active. Otherwise schedule a controlled drain investigation with explicit requeue and reconciliation rules.

Technical background: Celery documentation.

Keep the decision with the work.

Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.

Create an account See the product workflow