Laravel ShouldBeUnique: What Breaks Under Real Traffic

Laravel has two ways to stop a queued job running twice. ShouldBeUnique decides whether another copy may enter the queue, and WithoutOverlapping decides whether two copies may run at the same time. They fail in different ways, and most of the failures only appear once retries and worker crashes are involved. Left on their defaults, they can drop jobs without an error and leave behind locks that never expire.

ShouldBeUnique, ShouldBeUniqueUntilProcessing or WithoutOverlapping?#

Most of the mistakes below come from using the wrong one of these three. The unique locks act at dispatch and control what enters the queue; WithoutOverlapping acts at pickup and controls what executes.

Laravel documents them separately, under unique jobs and preventing job overlaps. They are different guarantees, and a job that needs both has to declare both.

Gotcha 1: A failed unique dispatch is silent#

When a ShouldBeUnique job can't acquire its unique lock, Laravel does not throw, does not log, and does not record a failed job. The job never enters the queue, so it never appears in Horizon.

That is correct for a dedupe gate, but a caller that expects feedback gets none. This line reads as if it queued a job:

RebuildSearchIndex::dispatch($accountId);

What it guarantees is that a PendingDispatch was created. Laravel may queue the job later, if it can acquire the unique lock.

On the normal dispatch() path, the unique lock is acquired in PendingDispatch::__destruct(), by way of shouldDispatch(), so the lock attempt and the push happen when that object is destroyed, not necessarily at the call site. Holding a reference defers both:

$pending = RebuildSearchIndex::dispatch($accountId);
// The unique lock may not have been attempted yet.
// The job may not have been pushed yet.

This timing affects any code that treats dispatch() as an immediate yes/no operation, and anything that depends on exceptions or object lifetime. With unique jobs, Foo::dispatch() returning does not tell you that Foo was queued, or even that Laravel has tried yet. If the lock is already held when Laravel checks, the job is discarded before it reaches the queue.

You can't rely on Horizon to show deduped jobs. Horizon observes queue events. A job rejected at dispatch never enters the queue, so Horizon has nothing to record. Laravel 13.25 added a UniqueJobSkipped event, which gives you something to listen for and log; on earlier versions the framework fires nothing at all.

A wrong uniqueId() loses work without an error. If you omit uniqueId(), Laravel's default discriminator is effectively empty, so uniqueness collapses to one job per class, globally:

class SyncCustomer implements ShouldQueue, ShouldBeUnique
{
    public function __construct(
        public int $customerId,
    ) {}
}

This looks per-customer, but without uniqueId() every SyncCustomer instance shares the same class-level lock, and a dispatch for customer 2 is discarded while customer 1's job holds it. Laravel does not include job arguments in the unique key. Put the business identity into uniqueId():

public function uniqueId(): string
{
    return (string) $this->customerId;
}

Gotcha 2: Unique jobs can still overlap#

ShouldBeUnique stops another copy from being queued while the unique lock exists; once the lock is released, another copy can be admitted. The lock is taken at dispatch and lives for uniqueFor seconds, so if uniqueFor is shorter than the job's full lifetime, the lock expires while the job is still running and a second copy can execute alongside it.

ShouldBeUniqueUntilProcessing allows overlap by design. It releases the unique lock when processing starts, so a new copy can enter the queue while the first one runs. That suits "latest state wins" jobs. Take recomputing a report for account 123: you don't want 500 waiting recomputes piling up, but a state change during a recompute should schedule the next one.

For that, pair it with WithoutOverlapping:

class RecomputeAccountReport implements ShouldQueue, ShouldBeUniqueUntilProcessing
{
    public int $uniqueFor = 1800;

    public function uniqueId(): string
    {
        return (string) $this->accountId;
    }

    public function middleware(): array
    {
        return [
            (new WithoutOverlapping("account-report:{$this->accountId}"))
                ->releaseAfter(60)
                ->expireAfter(900),
        ];
    }

    public function retryUntil(): DateTimeInterface
    {
        return now()->addMinutes(30);
    }
}

ShouldBeUniqueUntilProcessing keeps at most one copy waiting, and WithoutOverlapping keeps at most one copy running.

Set uniqueFor here too. The until-processing lock is held for less time than a normal ShouldBeUnique lock because it is released when work begins, but if the job sits in a stalled queue and is never picked up, a lock without a TTL stays held until someone deletes it.

Gotcha 3: Chains, batches and retries skip the unique lock#

The unique lock is acquired on the dispatch path only: PendingDispatch::shouldDispatch() calls UniqueLock::acquire(), and queued event listeners and scheduled jobs make the same call. Every other way of putting a job on a queue pushes it without the lock:

• The first job of a chain. Bus::chain() hands the head straight to the bus dispatcher, bypassing PendingDispatch. The later jobs are dispatched normally as each one finishes, so they do take the lock, and if it is held at that moment the rest of the chain is dropped without a trace.
• Batched jobs. Bus::batch() pushes through the queue's bulk() method, which knows nothing about ShouldBeUnique.
• Queue::push() and Queue::later(), and anything else that pushes a job or a raw payload directly.
• Retries. php artisan queue:retry and Horizon's retry button push the failed job's stored payload back with pushRaw().

// uniqueId() is honored: one copy is queued
ImportFeed::dispatch($feedId);

// uniqueId() is ignored: every job is queued, duplicates included
Bus::batch([
    new ImportFeed($feedId),
    new ImportFeed($feedId),
])->dispatch();

// Also ignored for the head of the chain
Bus::chain([new ImportFeed($feedId), new NotifyImportDone($feedId)])->dispatch();

Both are reported upstream, in #49263 for chains and #51798 for the general case, and the job batching docs cover the batch path.

The bypass is on the acquire side only. When a unique copy that skipped the lock finishes, CallQueuedHandler still releases the key. Laravel 13.24 made that release owner-checked, but a copy that never took the lock has no owner token, so UniqueLock::release() falls back to forceRelease() (on earlier versions every release is a force release). If a copy dispatched the normal way is holding the lock, the batched or chained copy frees it, and the next dispatch gets through as a duplicate. To keep uniqueness on these paths, enforce it before the push: acquire your own lock, or dedupe the list at the call site.

Gotcha 4: Unique locks are held across retries#

The lock is taken once, at dispatch, and it is not released when an attempt fails and the job returns to the queue. Laravel keeps it while the job remains in flight, which is usually what you want: a transient failure shouldn't let a duplicate in during the backoff window. A retry doesn't renew the lock either, so its TTL keeps counting down from the first dispatch.

That makes the uniqueFor sizing problem from Gotcha 2 worse, because the TTL has to cover every attempt and backoff, not one execution. On a job that can spend 20 minutes retrying through backoff, this is too short:

public int $uniqueFor = 120;

After two minutes the lock expires while the original job is still alive, a duplicate is admitted, and both can eventually run.

A long uniqueFor admits fewer duplicates, but an orphaned lock blocks the job for longer. A short one limits that, but allows duplicates during long retry windows. Size it from the job's lifecycle, retries included.

Gotcha 5: WithoutOverlapping can fail a job that never ran#

When a worker picks up a job, it increments the attempt count and checks the max-attempts ceiling before the body runs. Now add WithoutOverlapping. If the overlap lock is held, the middleware releases the contender back to the queue without calling handle(), but the pickup has already used an attempt. After enough collisions the job hits maxTries:

A job can fail with MaxAttemptsExceededException even though handle() was never invoked.

This follows from combining release-based middleware with low attempt limits, and the defaults make it worse. WithoutOverlapping defaults to releaseAfter = 0, so a blocked contender is released immediately. That creates a busy loop in which each pickup fails to acquire the lock, releases the job and uses another attempt. With tries = 1, the first collision fails the job:

Pickup #1: attempt is 1. Cannot acquire overlap lock. Released without running.
Pickup #2: attempt is 2. Worker sees max attempts exceeded. Job fails before handle().

A longer releaseAfter() slows this down:

(new WithoutOverlapping("account:{$this->accountId}"))
    ->releaseAfter(60);

It does not change the accounting. On a count-based tries budget, every overlap release still consumes an attempt. A 60-second delay spreads those attempts over minutes instead of milliseconds, and the job still fails without running once the budget is used up.

The fix is a time-based retry window with retryUntil():

public function retryUntil(): DateTimeInterface
{
    return now()->addMinutes(30);
}

With retryUntil() set, the worker's max-attempts check uses the clock instead of the counter, and retryUntil() supersedes the finite tries budget for this failure path. Overlap releases no longer count against a small fixed number. The job still fails once the timestamp passes, but not because the lock was unavailable three times before handle() ran. Waiting for a lock is not an application failure, and a time budget treats it that way.

Your job's backoff() does not control WithoutOverlapping releases either. Worker backoff applies to exception-driven releases; WithoutOverlapping calls release() with its own releaseAfter value. If you configured exponential backoff() and expected overlap contenders to follow it, they won't. Configure releaseAfter() explicitly for contention.

Setting both looks like this:

(new WithoutOverlapping("account:{$this->accountId}"))
    ->releaseAfter(60)
    ->expireAfter(900);

public function retryUntil(): DateTimeInterface
{
    return now()->addMinutes(30);
}

releaseAfter() controls how often a blocked job retries, and retryUntil() keeps those retries from exhausting the attempt budget. The same problem applies to any queue middleware that releases jobs before the body runs, including rate limiting.

Gotcha 6: A ShouldBeUniqueUntilProcessing job that never starts keeps its lock#

ShouldBeUniqueUntilProcessing releases its lock just before handle() runs. Because of that, CallQueuedHandler::failed() skips the release for these jobs, on the assumption that it already happened. The assumption breaks when the worker fails the job before it gets that far.

The worker does exactly that in its pickup check. If the job's retryUntil() deadline has passed, or its attempts are already over tries, the worker fails it before the handler sees it. Two ordinary situations lead there:

• A backlog outlasts retryUntil(). The deadline is resolved to a timestamp at dispatch, so a job that waits in a slow queue, or is delayed past its own window, is failed the moment a worker reaches it.
• Releases use up the tries. The WithoutOverlapping or RateLimited contention from Gotcha 5 counts attempts without running the body, and the unique lock stays held across those releases, so the pickup that finally exceeds the limit fails a job that still holds it.

The failed job's exception reads as if the job had been running and retrying:

Illuminate\Queue\MaxAttemptsExceededException:
App\Jobs\RecomputeAccountReport has been attempted too many times.

It never ran. Its lock is still held, and with the default uniqueFor of 0 it is held forever, so every later dispatch of that job for that uniqueId() is discarded. The job does show up once, in failed jobs. Retrying it from there happens to free the lock, because the retried copy releases the key when it starts, but nothing links that one failure to the dispatches that have gone missing since.

A deleted model can strand a lock on the same failure path. When a SerializesModels job's model is gone, recent Laravel versions release the lock from the job's hidden context on a normal pickup. A job refused at pickup goes through failed() instead, which has to unserialize the job to find its lock, throws ModelNotFoundException, and releases nothing, for plain ShouldBeUnique jobs as well.

Setting uniqueFor bounds all of this. If you can't, alert on MaxAttemptsExceededException failures of ShouldBeUniqueUntilProcessing classes and check for their laravel_unique_job:* key.

Gotcha 7: dontRelease() discards blocked jobs#

WithoutOverlapping offers dontRelease(), which changes how a blocked contender behaves. Instead of returning to the queue, the job is removed without running and is not retried. No failed job is recorded and no exception is thrown.

That is fine when the duplicate has no value ("only one refresh is needed; if another is already running, drop this one"). For work where every event must be processed, it loses events. Don't write this:

(new WithoutOverlapping($key))->dontRelease();

unless this sentence is true: "If this job collides with an existing job, losing it is semantically correct." A blocked job may be real work waiting for the lock rather than a duplicate. If every job matters, release it with a deliberate delay instead:

(new WithoutOverlapping($key))
    ->releaseAfter(60)
    ->expireAfter(900);

Gotcha 8: A lock with no TTL can outlive its worker#

Both unique locks and overlap locks are cache locks, and both can be created with no expiry. The unique-job default uniqueFor is 0; with Redis, that's a lock with no TTL. WithoutOverlapping's default expiresAfter is also 0, so it has no TTL either.

Normally the locks are released as PHP unwinds through finally blocks, queue handler cleanup and failure paths. That doesn't happen when a worker is killed. Processes time out, containers die, hosts run out of memory, and Horizon may escalate to a force-kill. A SIGKILL does not run your finally block. If a worker dies holding a lock with no TTL, the lock stays in the cache until someone deletes it.

That can block a job class permanently. For ShouldBeUnique, every future dispatch is discarded because the unique lock still exists. For WithoutOverlapping, future contenders keep releasing and using up attempts. Horizon tracks jobs, not cache locks, so clearing Horizon's pending jobs or purging its metadata does not necessarily remove laravel_unique_job:* or laravel-queue-overlap:* keys. Those belong to the cache store.

Set a TTL on both:

public int $uniqueFor = 1800;

(new WithoutOverlapping($key))->expireAfter(900);

The value matters. If it is too short, the lock expires while the first job is still running and another worker can acquire the same lock, so two copies run at once. If it is too long, a crashed job blocks the key for longer than necessary.

Set lock TTLs longer than the maximum legitimate holder lifetime, but not forever.

For unique jobs, that lifetime includes queue delay, attempts, backoff, releases and retries as well as runtime. For overlap locks, the TTL should exceed the job timeout and the longest realistic runtime, with some margin.

Key design is part of correctness#

Both mechanisms reduce part of your domain to a string key, and whether the lock protects anything depends on that string. For ShouldBeUnique, the key is the job class plus uniqueId(). For WithoutOverlapping, it is the middleware key, which Laravel prefixes with the job class name by default. Two job classes that pass the same key string therefore get different locks and do not exclude each other.

shared() turns that scoping off. The key is used as-is, so distinct job classes that pass the same key collapse onto one lock and exclude each other:

// ChargeAccount and RefundAccount must never run together for one account
(new WithoutOverlapping("account:{$accountId}"))->shared();

Without shared() here, a charge and a refund for the same account lock on ChargeAccount:account:123 and RefundAccount:account:123, which are different keys and give no protection. With it, both use account:123. Use shared() only when different job classes contend for the same resource; a single class protecting its own critical section should keep the default.

The common key mistakes:

• Missing uniqueId() creates class-global uniqueness.
• Keying on only an account id when the real invariant is account + integration makes unrelated work block each other.
• Forgetting shared() means two different job classes don't mutually exclude, even when they touch the same resource.
• Using shared() too broadly serializes unrelated classes and reduces throughput.
• Using an in-memory or per-node cache store makes uniqueness local to a process or machine.

Name the resource the key protects:

"stripe-sync:account:{$accountId}"   // good: scope and invariant are clear
"sync:{$id}"                          // bad: the protected resource is unclear

Encode the business invariant in the key, and use the same lock store for every producer and worker. Cache locks are only as shared and durable as the cache store behind them: if your dispatchers and workers don't use the same Redis or database-backed cache, the lock is not global.

Horizon doesn't change the semantics#

Horizon supervises and observes queues; it does not change any of these guarantees. It does not acquire unique locks, release overlap locks, inspect cache keys to explain dedupe, or clean orphaned Laravel cache locks when you clear Horizon queues. It shows symptoms: a job exhausted by overlap contention appears as a max-attempts failure, and a job deduped away by ShouldBeUnique appears nowhere. A dashboard retry may reset attempts, but it doesn't fix the contention that caused exhaustion.

TTLs matter more under Horizon, because it can terminate supervised workers forcefully. A worker killed while it holds a lock with no expiry never runs Laravel's cleanup code, so nothing ever releases that lock.

What Skyline changes#

We build Skyline, a drop-in replacement for Horizon, and several of the gotchas above are why. Since 1.4 it covers three of them:

• Stranded locks are released. When a ShouldBeUniqueUntilProcessing job is failed before it starts (Gotcha 6), Skyline releases its lock, but only while the lock is still held by the owner that job's own dispatch took it with. The failure says why, too: "never ran: its retryUntil() allowed 60s from dispatch, but a worker only picked it up 600s after dispatch" in place of "attempted too many times".
• Skipped dispatches are counted. Every dispatch discarded over a held lock is counted per job for the past hour and the past day, and the most recent are listed with their lock key — so a job that silently stopped running (Gotcha 1) shows up as a skip count that keeps climbing instead of as nothing at all.
• The dashboard lists held locks. A Locks & Limits screen shows the unique locks your dispatches hold, with its job, age and expiry, flags stranded ones, counts the dispatches each lock has skipped, and has a Release button that refuses to free a lock another dispatch has taken since the page loaded.

Unique job locks documents the release behaviour and its settings, and Locks & Limits the screen. The rest of the package is the Horizon you already run; Skyline vs Horizon lists what differs.

Rules worth standardizing#

Most teams can enforce these in code review:

• Always define uniqueId() for ShouldBeUnique jobs, unless class-global uniqueness is explicitly intended.
• Always set uniqueFor; a unique lock without it never expires.
• Size uniqueFor for the full lifecycle (queue delay, attempts, backoff and retries), since the lock is never renewed after dispatch.
• Don't expect ShouldBeUnique to hold for a chain's first job, Bus::batch(), Queue::push() or a retry; those skip the lock, so dedupe before pushing.
• Give ShouldBeUniqueUntilProcessing jobs a uniqueFor too; one failed before it starts keeps its lock.
• Always set expireAfter() for WithoutOverlapping; never rely on finally as your only cleanup path.
• Use releaseAfter() to control overlap retry cadence, not to fix attempt exhaustion.
• Prefer retryUntil() over low tries for jobs using release-based middleware; overlap releases are then limited by time rather than a small counter.
• Don't expect backoff() to affect WithoutOverlapping; configure releaseAfter() explicitly.
• Use a shared, durable cache store for locks. Array, local, or inconsistent stores are not distributed locks.
• Don't treat Horizon as a lock manager.
• Don't use dontRelease() unless losing the blocked job is semantically correct.

Locking is one part of a larger set of rules for queued jobs. Idempotency, atomicity, backoff, afterCommit and staying backwards compatible across a deploy are covered in 12 best practices for Laravel background jobs. If the problem is a queue that never drains rather than a job that never runs, the cause may be worker allocation instead of locking: Horizon queue balancing covers that.