Aglet

Track ready-to-claim age as its own job measure

The durable lesson is to measure the wait before work begins as a distinct interval. Document when a job is ready, what claim proves ownership, how lock and capacity waits appear, and which consumer should react. Keep runtime metrics separate so a handler change cannot mask a dispatch regression.

Keep the lesson for the next incident

  1. Define claim-age semantics

    Name enqueue, ready, claim, start, and terminal events, plus the queue and worker context attached to each. State the review condition for an eligible job that waits too long and how capacity or lock waits are classified.

  2. Retain dispatch fixtures

    Keep quiet, busy, lock-contended, and retry-created jobs with expected event order and interval relationships. Re-run them after changing polling, queue routing, worker pools, claim locks, eligibility, or retry behavior. Keep queue and worker identifiers attached so a later pool change cannot erase the boundary.

  3. Watch recurrence by boundary

    Review ready jobs with no poll, free workers that do not claim eligible work, and claim age that grows while runtime remains stable. Inspect queue and lock evidence before scaling workers. Assign ownership when a new dispatch path lacks the event contract.

What to carry forward

Preserve a ready-to-claim metric, event contract, and matched dispatch fixtures with capacity and lock context. Recurrence checks should identify the first waiting boundary. Reopen review when polling, routing, worker capacity, lock ownership, eligibility, or retry identity changes pre-execution behavior.

Technical background: Rails guides.

Keep the decision with the work.

Use a Work Item in Aglet to record the problem, the evidence you have, and the next decision. Add an owner and priority, then keep updates in the discussion so the next person can follow the reasoning.

Create an account See the product workflow