Laravel Job Retries and Backoff with ShouldBeUnique and WithoutOverlapping

When a queued job throws and has attempts left, the worker puts the same payload back on the queue with a delay taken from backoff. Nothing is dispatched again, so none of the checks that run at dispatch run again. ShouldBeUnique, ShouldBeUniqueUntilProcessing and WithoutOverlapping each do something different with their lock during that delay, and most of the surprises with these three happen there.

This post follows a job through the gap between two attempts for each mechanism, then covers two ways the retry schedule itself goes wrong: overlap releases that move a job further along its backoff array, and a backoff that runs past retryUntil(). Every behaviour here was checked with probe jobs on Laravel 13.24.

What the worker does between two attempts#

A worker increments the attempt count when it picks a job up. If handle() throws, the worker first checks whether this was the last attempt (attempts >= tries, or retryUntil() already passed). If it wasn't, it releases the job with a delay:

// Illuminate\Queue\Worker::calculateBackoff()
$backoff = $job->backoff() ?? $options->backoff;   // "10,60,300" or [10, 60, 300]

return $backoff[$job->attempts() - 1] ?? last($backoff);

The released job is the same payload with its attempt count raised. The constructor arguments are what they were at dispatch, and so are the unique lock's owner token and the retryUntil() timestamp. PendingDispatch doesn't run, so no unique lock is taken and none is renewed.

Backoff only applies to exceptions. A worker that is killed mid-job, by a timeout or the OOM killer, never reaches the release, and the job comes back when the connection's retry_after expires. The timeout and retry_after post covers that path. Everything below is about the exception path.

ShouldBeUnique keeps the lock for the whole retry schedule#

The lock is released in two places: after an attempt that finishes without being released, and in the failure handler when the job fails for good. A release for a retry is neither, so the lock stays held while the job waits. We dispatched a copy during a 60-second backoff and it was dropped, and a copy dispatched after the retry succeeded was queued. That is usually the behaviour you want.

The cost is that uniqueFor counts down from the dispatch and nothing extends it. It has to cover every attempt and every backoff, plus however long the job can sit in the queue:

class SyncInvoices implements ShouldQueue, ShouldBeUnique
{
    public $tries = 4;
    public $timeout = 120;

    public function backoff(): array
    {
        return [60, 300, 900];
    }

    // 1260s of backoff + 4 × 120s of runtime + room for a queue backlog
    public $uniqueFor = 3600;
}

With uniqueFor = 30 and a 60-second backoff, the lock expired while the first job was still waiting to retry. A second dispatch was queued, and both copies ran.

Laravel 13.24 made the release owner-checked, and that limits the damage. When the original job finally succeeded, its release used the owner token from its own dispatch, which no longer matched, so it left the second copy's lock alone and a third dispatch was dropped. On earlier versions every release was a force release: the original freed the second copy's lock and let a third one in.

ShouldBeUniqueUntilProcessing has nothing to hold during backoff#

ShouldBeUniqueUntilProcessing releases its lock just before handle() runs on the first attempt. If that attempt throws, the job waits out its backoff with no lock at all, and a new dispatch goes straight through. In our probe the queue then held two jobs: the retry, due in 60 seconds, and the fresh copy, due now.

"At most one waiting copy" therefore doesn't include copies waiting to retry. For a job that recomputes from current state that is harmless. It matters when the job carries its data in the constructor:

class PushPriceToChannel implements ShouldQueue, ShouldBeUniqueUntilProcessing
{
    public function __construct(
        public int $productId,
        public int $priceInCents,
    ) {}
}

Dispatch it with 1000, let the push time out once, and dispatch it again with 1200 during the backoff. The 1200 copy is available immediately and runs first. The 1000 copy runs when its backoff ends and overwrites the newer price. Nothing fails and nothing is logged. Pass the id alone and read the price in handle(), or use a debounced job (below), which drops the stale retry.

When the retry starts, it tries to release the lock again. On 13.24 that release uses the owner token from the first dispatch, so it leaves the fresh copy's lock alone: a third dispatch was dropped until the fresh copy started. Older versions force-released here too, so a retry could let a third copy in.

An overlap release behaves differently from an exception. If WithoutOverlapping sends the job back before handle() runs, Laravel keeps the unique lock, because the job never started. In our probe a dispatch made while the first copy was blocked on the overlap lock was dropped. That combination is what the ShouldBeUnique post recommends for "latest state wins" jobs, and it stays correct under contention. It stops holding once handle() throws.

WithoutOverlapping lets another copy in while the failed one waits#

The middleware is short enough to quote whole:

if ($lock->get()) {
    try {
        $next($job);
    } finally {
        $lock->release();
    }
} elseif (! is_null($this->releaseAfter)) {
    $job->release($this->releaseAfter);
}

The exception from handle() passes through the finally, so the overlap lock is gone before the worker has even calculated the backoff. Any copy that becomes available during the delay takes the lock and runs. In our probe copy A threw with a 10-second backoff, copy B was dispatched, and B ran before A's retry.

WithoutOverlapping promises that two copies won't run at the same time. It doesn't promise they run in dispatch order, and a failing job is exactly the one that gets reordered. If order matters, as with applying ledger entries for one account, a job has to check its preconditions in handle(), or the work for that key has to go through something that is ordered, such as a chain or a queue with a single worker.

Overlap releases move a job along its backoff array#

A job blocked by WithoutOverlapping is released after the worker has already counted the attempt. The ShouldBeUnique post covers how that uses up tries. It also changes the backoff, because the backoff is picked by attempt number:

backoff() = [10, 60, 300], releaseAfter(5)

Attempt 1: overlap lock held, released for 5s
Attempt 2: overlap lock held, released for 5s
Attempt 3: handle() throws, backoff[3 - 1] = 300s

The first real failure waits five minutes instead of ten seconds. Under heavy contention every exception lands on last($backoff). So write the array assuming any entry can be the first one used, or return a single value from backoff() for jobs behind release-based middleware. RateLimited releases the same way and shifts the index the same way.

A backoff that runs past retryUntil() loses the exception#

retryUntil() is turned into a timestamp at dispatch. When handle() throws, the worker fails the job at once only if that timestamp has already passed. Otherwise it releases the job with the full backoff, even if the backoff ends after the deadline.

public $backoff = 120;

public function retryUntil(): DateTimeInterface
{
    return now()->addSeconds(60);
}

Our probe job threw on its first attempt and was released for 120 seconds. When a worker picked it up, the deadline had passed, and the worker failed it before handle() ran:

Illuminate\Queue\MaxAttemptsExceededException:
App\Jobs\DeadlineJob has been attempted too many times.

That is the exception on the failed job, and the one your failed() method receives. The exception that actually broke the job was reported when it was thrown, so it is in your logs or error tracker, but nothing connects it to the failed job. Keep the sum of the backoffs inside the retryUntil() window, with room for queue wait, so the last attempt runs and fails with its own exception.

On a ShouldBeUniqueUntilProcessing job, a pickup failure on the first attempt also strands the unique lock. That case is covered in Gotcha 6 of the ShouldBeUnique post. On later attempts it can't happen, because the lock was released when the first attempt started.

Debounced jobs drop the retry instead#

Laravel 13.6 added #[DebounceFor]. Every dispatch overwrites an owner token in the cache and is queued with a delay, and when a worker picks up a job whose token is no longer current, it deletes the job and fires JobDebounced. The check runs on every pickup, retries included:

#[DebounceFor(30)]
class PushPriceToChannel implements ShouldQueue
{
    public function debounceId(): string
    {
        return (string) $this->productId;
    }
}

In our probe the first copy threw and was released for 60 seconds, and a second dispatch was made during that backoff. When the retry came up it was deleted without running, and only the newer copy ran. That is the behaviour the price example above needs. A debounced job can't also implement ShouldBeUnique: Laravel throws a LogicException at dispatch.

Sizing the numbers together#

Each of these settings is usually chosen on its own, and most of the problems above come from them disagreeing. Work out the retry window first and derive the rest from it:

class RecomputeAccountReport implements ShouldQueue, ShouldBeUniqueUntilProcessing
{
    public $timeout = 120;

    public function backoff(): int
    {
        return 60;                          // flat: overlap releases can't shift it
    }

    public function retryUntil(): DateTimeInterface
    {
        return now()->addMinutes(30);       // the retry window
    }

    public $uniqueFor = 2400;               // window + time a backlog can hold it

    public function middleware(): array
    {
        return [
            (new WithoutOverlapping("account-report:{$this->accountId}"))
                ->releaseAfter(60)
                ->expireAfter(180),         // > $timeout, < retry_after (300)
        ];
    }
}

• uniqueFor is longer than the retry window plus a realistic queue wait, because the lock is never renewed.
• The backoffs add up to less than the retryUntil() window, so the last attempt fails with its own exception.
• backoff is flat, or written so that any entry can come first, if the job sits behind WithoutOverlapping or RateLimited.
• expireAfter is longer than timeout, and shorter than retry_after, so a killed worker's lock is gone before its job comes back.

Seeing the gap in a dashboard#

Horizon shows a job waiting out its backoff in the same delayed list as a job dispatched with delay(), and a failed job carries only its last exception. Skyline, the Horizon replacement we build, lists retries separately from scheduled jobs, can run a retrying job immediately once you have shipped the fix, and records the exception from each earlier attempt in a Previous Attempts panel, so the exception behind a MaxAttemptsExceededException stays attached to the job. Its Locks & Limits screen shows which unique locks are held while a job waits.

The ShouldBeUnique post covers the rest of the lock behaviour, including chains, batches and locks without a TTL, and the queue job lifecycle follows a job from dispatch to deletion.