Laravel Job Uniqueness Controls: What Breaks Under Real Traffic

Laravel's queue abstractions make concurrency feel deceptively simple: mark a job ShouldBeUnique and move on. In production, the edge cases are where the real work is. ShouldBeUnique and WithoutOverlapping solve different problems and fail in different ways. Left on their defaults, they quietly produce dropped jobs, exhausted attempts, orphaned locks, and invisible bottlenecks.

This article walks through the queue-locking gotchas that only surface once real traffic, retries and worker crashes enter the picture.

ShouldBeUnique vs. WithoutOverlapping: which one do you need?#

Almost every mistake in this article traces back to reaching for the wrong one of these three, so it's worth pinning down the differences before the gotchas. The short version: the unique locks act at dispatch and control what enters the queue; WithoutOverlapping acts at pickup and controls what executes.

Laravel documents them separately — unique jobs and preventing job overlaps — and that separation is the point: they are different guarantees, and a job that needs both must declare both.

Gotcha 1: A failed unique dispatch is silent#

When a ShouldBeUnique job can't acquire its unique lock, Laravel does not throw, does not log, and does not record a failed job. The job never appears in Horizon and never enters the queue. It is simply not dispatched.

A unique-dispatch collision is a silent no-op.

That's correct behavior for a dedupe gate, but it's a bad surprise if the caller expects feedback. Code like this can lie by omission:

RebuildSearchIndex::dispatch($accountId);

That line does not mean "a job was queued." It means "a PendingDispatch was created, and Laravel may queue the job later, if the unique lock can be acquired."

The "later" matters. On the normal dispatch() path, the unique lock is acquired from PendingDispatch::__destruct(), by way of shouldDispatch(). Lock acquisition and the actual push don't necessarily happen at the call site. Holding a reference can defer the attempt entirely:

$pending = RebuildSearchIndex::dispatch($accountId);
// The unique lock may not have been attempted yet.
// The job may not have been pushed yet.

That timing interacts with exceptions, object lifetime, and any code that assumes dispatch() is an immediate yes/no operation. Foo::dispatch() is not a reliable signal that Foo was queued — with unique jobs, it may not have tried yet. If the lock is already held when Laravel finally checks, the job disappears before it's ever queue-visible.

Two practical consequences follow.

You can't rely on Horizon to show deduped jobs. Horizon observes queue events. A job rejected at dispatch never enters the queue, so it leaves no trace.

A bad uniqueId() causes invisible data loss. If you omit uniqueId(), Laravel's default discriminator is effectively empty, collapsing uniqueness to one job per class, globally:

class SyncCustomer implements ShouldQueue, ShouldBeUnique
{
    public function __construct(
        public int $customerId,
    ) {}
}

This looks per-customer. It isn't. Without uniqueId(), every SyncCustomer instance shares the same class-level lock, and customer 2 dedupes away behind customer 1. Laravel does not include job arguments in the unique key for you. Put the real business identity into uniqueId():

public function uniqueId(): string
{
    return (string) $this->customerId;
}

Gotcha 2: Unique jobs can still overlap#

This feels contradictory until you separate dispatch from execution. ShouldBeUnique stops another copy from being queued while the unique lock exists; once it's released, another can be admitted. The lock is taken at dispatch and lives for uniqueFor seconds — so if uniqueFor is shorter than the job's real lifetime, the lock expires out from under a still-running job and a second copy can execute alongside it.

ShouldBeUniqueUntilProcessing makes this obvious: its entire purpose is to release the unique lock the moment processing starts, which lets a new copy enter the queue while the first copy runs. That's often exactly what you want for "latest state wins" jobs. Consider recomputing a report for account 123: you don't want 500 waiting recomputes piling up, but while one recompute runs, a new state change should schedule the next one.

The right shape pairs both mechanisms:

class RecomputeAccountReport implements ShouldQueue, ShouldBeUniqueUntilProcessing
{
    public int $uniqueFor = 1800;

    public function uniqueId(): string
    {
        return (string) $this->accountId;
    }

    public function middleware(): array
    {
        return [
            (new WithoutOverlapping("account-report:{$this->accountId}"))
                ->releaseAfter(60)
                ->expireAfter(900),
        ];
    }

    public function retryUntil(): DateTimeInterface
    {
        return now()->addMinutes(30);
    }
}

Each mechanism does a different job. ShouldBeUniqueUntilProcessing collapses the waiting backlog; WithoutOverlapping protects the critical section at runtime.

uniqueFor still matters here. The until-processing lock has a smaller exposure window than a normal ShouldBeUnique lock because it's released when work begins — but if the job sits unpicked in a stalled queue, a no-TTL lock is still a no-TTL lock. Set uniqueFor even with ShouldBeUniqueUntilProcessing; smaller risk is not zero risk. If you need both "only one waiting" and "only one running," you need both mechanisms.

Gotcha 3: Batch dispatch bypasses the unique lock entirely#

The unique lock is acquired in exactly one place: the single-dispatch path, where PendingDispatch::shouldDispatch() calls UniqueLock::acquire(). Jobs added through Bus::batch() never take that path. They're pushed via the queue's bulk() method, which knows nothing about ShouldBeUnique.

ShouldBeUnique is a silent no-op for jobs dispatched inside a batch.

// uniqueId() is honored — one copy queued
ImportFeed::dispatch($feedId);

// uniqueId() is ignored — every job is queued, duplicates and all
Bus::batch([
    new ImportFeed($feedId),
    new ImportFeed($feedId),
])->dispatch();

The same applies to anything that reaches Queue::bulk() or pushes raw payloads directly instead of going through dispatch() — see job batching for what the batch path actually does. The bypass is on the acquire side only: when a batched unique job runs, CallQueuedHandler still calls release() on the key. That's a harmless no-op on a lock that was never taken, but it means you can't lean on the release path to paper over the missing acquire. If you need uniqueness inside a batch, enforce it yourself before adding the job — acquire your own lock, or dedupe the list at the call site.

Gotcha 4: Unique locks are held across retries#

The lock is taken once, at dispatch — and it is not released when an attempt fails and the job returns to the queue. Laravel keeps it while the job remains in flight, which is usually what you want: a transient failure shouldn't let a duplicate slip in during the backoff window. But a retry never renews the lock either, so its TTL keeps counting down from that first dispatch.

That widens the uniqueFor-sizing trap from Gotcha 2: the TTL has to cover the whole retry lifecycle, not one execution attempt. This is wrong on a job that can spend 20 minutes retrying through backoff:

public int $uniqueFor = 120;

After two minutes the lock expires while the original job is still alive, a duplicate is admitted, and both can eventually run.

The trade-off is real. A long uniqueFor reduces duplicate admission but increases the blast radius of an orphaned lock; a short one shortens that exposure but allows duplicates during long retry windows. There's no magic value — size it from the job's actual lifecycle, retries included.

Gotcha 5: WithoutOverlapping can fail a job that never ran#

When a worker picks up a job, it increments the attempt count and checks the max-attempts ceiling before the body runs. Now add WithoutOverlapping. If the overlap lock is held, the middleware releases the contender back to the queue without calling handle() — but the pickup still consumed an attempt. Repeat a few times and the job hits maxTries:

A job can fail with MaxAttemptsExceededException even though handle() was never invoked.

It's the natural outcome of release-based middleware combined with low attempt limits, and the defaults make it worse. WithoutOverlapping defaults to releaseAfter = 0, so a blocked contender is released immediately, creating a busy loop: pick up, fail to acquire, release, pick up again, burn another attempt. With tries = 1, the first collision is fatal:

Pickup #1: attempt is 1. Cannot acquire overlap lock. Released without running.
Pickup #2: attempt is 2. Worker sees max attempts exceeded. Job fails before handle().

A longer releaseAfter() helps only partially:

(new WithoutOverlapping("account:{$this->accountId}"))
    ->releaseAfter(60);

This slows the bounce loop but doesn't change the accounting model. On a count-based tries budget, every overlap release still consumes an attempt; a 60-second delay spreads those attempts over minutes instead of milliseconds, but the job can still fail without ever running once the budget is exhausted. releaseAfter() slows exhaustion. It does not fix it.

The real fix is a time-based retry window with retryUntil():

public function retryUntil(): DateTimeInterface
{
    return now()->addMinutes(30);
}

With retryUntil() set, the worker's max-attempts check is governed by the clock instead of the counter. The key point isn't just "more time" — it's that retryUntil() supersedes the finite tries budget for this failure path. Overlap bounces no longer burn through a small fixed count. The job can still expire when the timestamp passes, but it won't fail simply because the mutex was unavailable three times before handle() ran. That maps to reality: a mutex wait is not an application failure, so don't model it with a tiny fixed attempt budget.

Watch the backoff trap. Your job's backoff() does not control WithoutOverlapping releases. Worker backoff applies to exception-driven releases; WithoutOverlapping calls release() with its own releaseAfter value. If you configured exponential backoff() and expected overlap contenders to follow it, they won't. Configure releaseAfter() explicitly for contention.

A safer shape combines both controls:

(new WithoutOverlapping("account:{$this->accountId}"))
    ->releaseAfter(60)
    ->expireAfter(900);

public function retryUntil(): DateTimeInterface
{
    return now()->addMinutes(30);
}

Use releaseAfter() to control contention cadence; use retryUntil() to avoid attempt-budget exhaustion. They solve different parts of the problem. The same release-burns-attempts risk applies to any queue middleware that releases jobs before the body runs, including rate limiting.

Gotcha 6: dontRelease() is a silent discard#

WithoutOverlapping offers dontRelease(), which changes how a blocked contender behaves. Instead of returning to the queue, the job falls through without running and without being retried. No failed job, no exception — the job is simply gone.

dontRelease() is a silent discard.

That's fine when overlap means the duplicate is genuinely useless ("only one refresh is needed; if another is already running, drop this one"). It's dangerous for business-critical work where every event must be processed. Don't write this:

(new WithoutOverlapping($key))->dontRelease();

unless this sentence is true: "If this job collides with an existing job, losing it is semantically correct." If every job matters, release it with a deliberate delay instead:

(new WithoutOverlapping($key))
    ->releaseAfter(60)
    ->expireAfter(900);

A blocked job is not necessarily a duplicate. Sometimes it's real work waiting for a lock.

Gotcha 7: No-TTL locks are operational debt#

Both unique locks and overlap locks are cache locks, and both can be created with no expiry. The unique-job default uniqueFor is 0; with Redis, that's a lock with no TTL. WithoutOverlapping's default expiresAfter is also 0. Again, no TTL.

Normally locks are released by PHP unwinding cleanly through finally blocks, queue handler cleanup, and failure paths. Production doesn't always unwind cleanly. Workers get killed, processes time out, containers die, hosts OOM, and Horizon may escalate to a force-kill. A SIGKILL does not run your finally block — PHP never gets to clean up. If a worker dies holding a no-TTL lock, the lock can live forever.

No-TTL queue locks can permanently wedge a job class.

For ShouldBeUnique, future dispatches are silently dropped because the unique lock still exists. For WithoutOverlapping, future contenders keep releasing and burning attempts. And Horizon won't save you: it tracks jobs, not cache locks. Clearing Horizon's pending jobs or purging its metadata does not necessarily remove laravel_unique_job:* or laravel-queue-overlap:* cache keys — those belong to the cache store.

The baseline rule is simple:

public int $uniqueFor = 1800;

(new WithoutOverlapping($key))->expireAfter(900);

But the number matters. Too short, and the lock expires while the first job is still running, letting another worker acquire the same lock — real overlap. Too long, and a crashed job blocks the key longer than necessary. So:

Set lock TTLs longer than the maximum legitimate holder lifetime, but not forever.

For unique jobs, that lifetime isn't just runtime — it includes queue delay, attempts, backoff, releases, and retries. For overlap locks, it should exceed the job timeout and realistic max runtime, with margin.

Key design is part of correctness#

Every lock-based mechanism eventually reduces your domain to a string key, and that string becomes part of your correctness model. For ShouldBeUnique, the key is the job class plus uniqueId(). For WithoutOverlapping, it's the middleware key — but by default Laravel scopes that key to the job class, internally prefixing it with the class name. Two different job classes that pass the same key string therefore get different locks and don't actually exclude each other.

shared() turns that scoping off. The key is used as-is, so distinct job classes that pass the same key collapse onto one lock and run mutually exclusive:

// ChargeAccount and RefundAccount must never run together for one account
(new WithoutOverlapping("account:{$accountId}"))->shared();

Without shared() here, a charge and a refund for the same account effectively lock on ChargeAccount:account:123 and RefundAccount:account:123 — different keys, no protection. With it, both collapse onto account:123. Reach for shared() only when distinct job types genuinely contend over the same physical resource; a single class protecting its own critical section wants the default.

With that established, the common mistakes are predictable:

• Missing uniqueId() creates class-global uniqueness.
• Keying on only an account id when the real invariant is account + integration makes unrelated work block each other.
• Forgetting shared() means two different job classes don't mutually exclude, even when they touch the same resource.
• Using shared() too broadly serializes unrelated classes and collapses throughput.
• Using an in-memory or per-node cache store makes uniqueness local to a process or machine.

A lock key is a production API. Treat it like one. Name the resource being protected:

"stripe-sync:account:{$accountId}"   // good — scope and invariant are obvious
"sync:{$id}"                          // bad — protects what, exactly?

Make the scope obvious, encode the business invariant, and keep the same lock store across all producers and workers. Cache locks are only as shared and durable as the cache store behind them: if your dispatchers and workers don't use the same Redis or database-backed cache, your lock is not global.

Horizon doesn't change the semantics#

Horizon is excellent at supervising and observing queues, but it doesn't rewrite these guarantees. It does not acquire unique locks, release overlap locks, inspect cache keys to explain dedupe, or clean orphaned Laravel cache locks when you clear Horizon queues. It shows symptoms: a job exhausted by overlap contention appears as a max-attempts failure; a job deduped away by ShouldBeUnique appears nowhere. A dashboard retry may reset attempts, but it doesn't fix the contention that caused exhaustion.

Horizon makes queue behavior visible. It does not make lock behavior safe. If anything, it makes TTL discipline more important, because supervised workers can be terminated forcefully. Kill a worker while it holds a no-expiry lock and Laravel never runs its cleanup code. Under Horizon, no-TTL locks aren't a harmless default — they're an outage waiting for the right timeout.

This gap is the reason Skyline surfaces unique job locks as a first-class thing you can see and clear from the dashboard, rather than a Redis key you have to know to go looking for.

Rules worth standardizing#

Most teams should turn these into code-review rules:

• Always define uniqueId() for ShouldBeUnique jobs, unless class-global uniqueness is explicitly intended.
• Always set uniqueFor; never rely on an immortal unique lock.
• Size uniqueFor for the full lifecycle — queue delay, attempts, backoff, and retries — since the lock is never renewed after dispatch.
• Don't expect ShouldBeUnique to hold inside Bus::batch(); batched jobs skip the lock, so dedupe before adding them.
• Always set expireAfter() for WithoutOverlapping; never rely on finally as your only cleanup path.
• Use releaseAfter() to control overlap retry cadence, not to fix attempt exhaustion.
• Prefer retryUntil() over low tries for jobs using release-based middleware; the overlap bounce is then governed by time rather than a small counter.
• Don't expect backoff() to affect WithoutOverlapping; configure releaseAfter() explicitly.
• Use a shared, durable cache store for locks. Array, local, or inconsistent stores are not distributed locks.
• Don't treat Horizon as a lock manager. It isn't one.
• Don't use dontRelease() unless losing the blocked job is semantically correct.

The core takeaway#

Laravel's queue primitives are good, but their names are easy to over-read. ShouldBeUnique gives you admission control — "should another copy of this job be admitted to the queue?" WithoutOverlapping gives you runtime exclusion — "may two copies execute at once?" They are not substitutes; they compose, and the highest-quality queue code is explicit about which boundary it protects. The highest-risk code says "make this unique" and leaves the rest to defaults. The defaults are convenient. They are not a production concurrency policy.

Locking is one slice of a larger contract. The rest of it — idempotency, atomicity, backoff, afterCommit, and staying backwards compatible across a deploy — is in 12 best practices for Laravel background jobs. And if the symptom you're actually chasing is a queue that never drains rather than a job that never runs, the cause may be worker allocation instead of locking: Horizon queue balancing covers that.