Reliability playbooks
Reason about failures, retries, delays, and incomplete processing. Choose a scenario, then the decision you need to make.
Intermittent errors
Requests fail only some of the time, making a workflow difficult to reproduce and leaving the boundary between transient and persistent errors unclear.
Job retries
Background work retries after failure, raising questions about idempotency, side effects, retry limits, and whether the final outcome is trustworthy.
Queue backlog
Queued work accumulates faster than it is completed, delaying customer-visible outcomes and making age, capacity, and failure isolation unclear.
Request timeouts
A request exceeds a deadline before the client or server can establish whether work completed, leaving retries and partial state risky.
Stale results
A screen or API returns older data after newer input or state exists, leading people to make decisions from results that no longer represent the requested view.
Duplicate processing
One logical action is processed more than once, creating repeated effects or conflicting updates when retries, concurrency, or replays overlap.
Missing events
An expected event disappears between creation and consumption, leaving projections, notifications, or downstream work without evidence that a change occurred.
Out-of-order events
Related events arrive in a different sequence than they were created, allowing an older update to overwrite newer state or confuse downstream work.
Rate-limit responses
Requests receive rate-limit responses, raising questions about scope, reset behavior, client retry safety, fairness, and whether critical work can continue.
Failing scheduled work
Work intended to run on a schedule fails, runs late, overlaps, or misses a window, leaving recurring customer and operational outcomes incomplete.