13 posts · Updated · Boring Observability
Dispatching a Laravel job is one line. Most of what goes wrong happens after it, between a worker reserving the job and the job being deleted, and it tends to surface only once there is enough traffic for two workers to touch the same thing.
These posts follow a job through that gap. The life of a queued job has the whole state machine in one place. The others each take one setting or feature (retry_after, ShouldBeUnique, backoff, chains, Horizon's balancing) and show what it does when the assumptions behind it stop holding.
For changes to the queue in the framework itself, the Laravel queue changelog lists every queue-related commit since Laravel 13.0.
· 13 min read
A vendor API answers 429 and says how long to wait, and Laravel ignores it. Reading Retry-After in every form vendors send, releasing a job without spending its tries, and a job middleware that makes one 429 pause every job for that API.
Read more· 10 min read
Dispatch 300 five-job chains on 10 workers and the first one finishes after 40 minutes, not two. Where the framework queues each next link, two ways to keep chains in order without the wait, and how a ShouldBeUnique link silently ends other chains.
Read more· 14 min read
Horizon ships one notification, and it cannot fire when Horizon is down. Which alert rules Spatie's Prometheus gauges support, which ones need a counter exporter, how to carry them into Grafana, and the failures that never show up in a metric at all.
Read more· 9 min read
A retried job is released, not dispatched again, so each lock behaves differently while it waits out its backoff. What ShouldBeUnique, ShouldBeUniqueUntilProcessing and WithoutOverlapping do in that gap, and how overlap releases and retryUntil() bend the backoff itself.
Read more· 11 min read
A job dispatched inside a transaction can reach a worker before its rows commit. Where to turn on afterCommit, how it behaves with nested transactions, deadlock retries and ShouldBeUnique, and why Queue::fake() won't catch a rollback.
Read more· 13 min read
Horizon deletes its throughput hash on every snapshot, so a Prometheus exporter that reads it loses the jobs processed between the last scrape and the reset. Why the existing exporters undercount, and a two-file counter exporter with the PromQL and alert rules to go with it.
Read more· 13 min read
Clearing a Laravel queue deletes the payloads but not the ShouldBeUnique locks they hold, and with the default uniqueFor those locks never expire. How to see what is in a backlog, what queue:clear deletes, and how to drain it without leaving a job that can't be dispatched again.
Read more· 24 min read
For most of its life a queued job is a JSON string moving between a Redis list and two sorted sets. The lifecycle in order: what dispatch writes, why exactly one worker gets it, why the attempt is spent at pop, and how a job survives a deploy.
Read more· 13 min read
Two numbers in two config files decide whether a job runs once or twice, and nothing in Laravel checks their order. What retry_after leases and why equal values already fail. Sometimes the symptom is a failure recorded for work that succeeded.
Read more· 16 min read
"60 requests a minute" and "no parallel requests" are different limits, handled in different places in Laravel. RateLimited, WithoutOverlapping, Redis funnels and single-worker supervisors compared, and how to let urgent jobs go first without breaking the limiter.
Read more· 11 min read
Many job bugs come from assuming a job runs once, on time, on the code that dispatched it. In development that holds; under production load it doesn't. Twelve practices for Laravel queued jobs, ending with what happens to queued payloads when you deploy a change to the job class.
Read more· 10 min read
Every Horizon supervisor makes you choose: turn balancing on and pay for a floor of idle workers, or turn it off and let a busy queue starve the ones listed after it. How both costs grow with the number of queues, and how Skyline's weighted queues avoid them.
Read more· 15 min read
ShouldBeUnique and WithoutOverlapping solve different problems and fail in different ways. What each one locks and when, and the failures that show up only at volume: dispatches dropped without an error, chains and retries that skip the lock, and locks that never expire.
Read more