Skyline

Cache Stampedes in Laravel: remember(), Locks and Cache::flexible() Under Load

· 10 min read · Boring Observability

Verified against Laravel 13.42 · Redis 7.4 · predis 3.7 · phpredis 6.3

When a cached value expires, Cache::remember() runs its closure again for the next request that asks for it, and for every request that arrives before that first one has finished. A value that takes a second to work out, read 50 times a second, gets worked out 51 times at once. This post runs remember(), a lock and Cache::flexible() under the same load. A lock cuts the work to one build, and every request that arrives during it still waits. flexible() is the only one where no request waits, for as long as the key is still cached, and a lock is the only one that protects a key that is gone. Put together, they cover both.

The numbers come from a fresh Laravel 13.42 app under php-fpm with 100 workers behind nginx, with Redis 7.4 as the cache. The cached value takes one second to work out and lives for five, and requests arrive 50 times a second, evenly spaced. predis and phpredis gave the same results. When the value's time was up:

Value builtRequests that waited for itOldest value served
remember()51 times51, a second each5 s
remember() + lockonce51, up to 1.3 s5 s
flexible(), key cachedonce, after a responsenone, slowest 85 ms6 s
flexible(), key gone53 times53, a second each
flexible() + lock, key goneonce53, up to 1.3 s, then none

The lock spares whatever the closure reads from, and leaves the users where remember() had them. flexible() spares both, by serving the value past its fresh time for as long as the rebuild takes, until the key is gone altogether.

What Cache::remember() does when a key expires#

remember() is a read, the closure and a write, with nothing between them:

$report = Cache::remember('report', 300, fn () => Report::build());

// Illuminate\Cache\Repository, in short:
$value = $this->get($key);

if (! is_null($value)) {
    return $value;
}

$value = $callback();

$this->put($key, $value, $ttl);

Between the get() that misses and the put() that writes, every other request that reads the key misses too and starts its own $callback(). Nothing tells them another process is already on it. In the second after the key expired, 51 requests ran the closure, and each of them took just over a second to answer. The requests around them were served from the cache in 7 ms at the median.

With a ten-minute TTL instead of five seconds, the same wave comes once every ten minutes. Over those ten minutes the hit ratio reads 99.8%, and whatever the closure reads from still answered the same question 51 times at once.

Every key at once: deploys and cache:clear#

An expiry stampedes one key. Emptying the cache stampedes every hot key together, each with a wave of its own. php artisan cache:clear does that, and so does php artisan optimize:clear, which runs cache:clear along with clearing the config, route and view caches. A deploy script that calls optimize:clear to be safe sends the first seconds of traffic on the new release to whatever the closures read from. php artisan optimize rebuilds the framework's caches and leaves the application's alone.

When a deploy changes the shape of a cached value, a new key name such as report:v2 lets the old entries expire on their own. Keys written together, by a job that warms ten thousand products with the same TTL, also expire together; a few random seconds on each TTL, now()->addSeconds(3600 + random_int(0, 300)), spreads them out.

One process works it out, the others wait#

use Illuminate\Support\Facades\Cache;

$report = Cache::get('report') ?? Cache::withoutOverlapping(
    'report:lock',
    fn () => Cache::remember('report', 300, fn () => Report::build()),
    lockFor: 10,
    waitFor: 5,
);

withoutOverlapping(), in Laravel 12.47 and later, takes an atomic lock, waits up to waitFor seconds for it, runs the closure and releases it. On older versions the same thing is Cache::lock('report:lock', 10)->block(5, fn () => ...). The get() in front keeps the lock off the path of every hit, so it is only taken on a miss. The remember() inside reads the key again once a process holds the lock: the first process to get it finds nothing and builds the report, and every process after it finds the report and returns it.

Under the same load this ran the closure once per expiry. The 50 requests that arrived while it ran waited between 258 ms and 1,270 ms, 761 ms at the median. The last of them waited longer than the build itself, because block() tries the lock every 250 ms and sleeps in between; betweenBlockedAttemptsSleepFor() on a Cache::lock() shortens the gap. Each waiting request holds a php-fpm worker while it sleeps, so the lock saves the work behind the cache and leaves the number of busy workers where the stampede had it.

Both durations have to fit the build:

  • waitFor shorter than the build throws LockTimeoutException for every request that waited that long. With a three-second build and waitFor: 2, 30 of 80 requests got it. Catch it and serve something, or wait longer.
  • lockFor shorter than the build lets the lock expire while the first process is still working, and the next waiter takes it and starts again. With a three-second build and lockFor: 1, three builds started, a second apart.
  • lockFor: 0, withoutOverlapping()'s default, is a lock with no expiry. A process killed while holding it never releases it. After one was killed, every request for the key waited its five seconds and failed with LockTimeoutException, 80 of 80, until the lock was deleted by hand.

Serving the old value while one process refreshes: Cache::flexible()#

$report = Cache::flexible('report', [300, 900], fn () => Report::build());

flexible(), in Laravel 11.23 and later, is Laravel's stale-while-revalidate. The first number is how long the value counts as fresh. The second is how long the key lives, counted from the same write: [300, 900] serves the value fresh for five minutes and stale for ten more. Beside the value it stores the time it was written, under illuminate:cache:flexible:created:report.

A read inside the fresh window returns the value. A read in the stale window returns the stale value too, and registers a deferred function to rebuild it after the response has been sent. The rebuild takes a lock, illuminate:cache:flexible:lock:report, so one process runs it however many requests saw the value stale. A read that finds no value at all runs the closure there and then, as remember() does.

With [5, 30] and the key already cached, no request ran the closure in twelve seconds. Two rebuilds ran after responses, and the slowest response took 85 ms. The price was age: the oldest value served was six seconds old, the five seconds of freshness plus the second the rebuild took. Under php-fpm the request that triggers a rebuild gets its response first, and then its worker spends the next second rebuilding.

When flexible() stampedes anyway#

flexible() spares the requests that find a stale value. Two situations leave them nothing to find, and both stampede exactly like remember():

  • A cold key. After a deploy that clears the cache, or the first time a key is read, 53 requests ran the closure in the first second.
  • A key nobody read during its stale window. With [2, 4] and six quiet seconds, the next burst found the key gone, and 54 requests ran the closure.

The second number fixes the second case. It costs only memory, so it can run to hours where the first runs to minutes, long enough that a quiet night does not empty the key. A cold key needs a lock.

flexible() with a lock for a missing key#

$report = fn () => Cache::flexible(
    'report',
    [300, 900],
    fn () => Report::build(),
    lock: ['seconds' => 30],
);

$value = Cache::get('report') === null
    ? Cache::withoutOverlapping('report:lock', $report, lockFor: 10, waitFor: 5)
    : $report();

A cached key, fresh or stale, goes straight to flexible(). A missing one takes the lock first. The first process to get it builds the value through flexible(), which stores it with its timestamp, and the processes waiting behind it find it fresh. Started cold under the same load, it built the value once, and the other 52 requests in that first second waited between 259 ms and 1,296 ms, as they do behind the lock alone. From then on no request waited, and the value was rebuilt twice in the background in twelve seconds. After a quiet spell that outlived [2, 4], it built the value once, where flexible() alone built it 54 times. The cost is a second read on every request, the get() in front.

That makes it the one to use when a value may be a few seconds old. When it may not, as with a balance or a stock count, remember() behind a lock never serves an old value, and each expiry makes the requests that arrive during the build wait for it. Warming the key from an artisan command in the deploy script, after anything that clears the cache and before traffic reaches the new release, spares even the first wait: in a command the closure runs inline on a miss and the value is stored.

When the rebuild doesn't run#

Deferred functions run once the work in front of them has finished, and only if it went well:

  • After an HTTP response with a status below 400. A stale read in a request that ended in a 500 left the value as it was, and the next request that returned 200 rebuilt it.
  • After a queued job that succeeded. A job that read the stale value and then failed rebuilt nothing.
  • After an artisan command that exits with 0. A command that called flexible() in a loop for 16 seconds got no rebuild while it ran: the value it read aged to ten seconds and was then rebuilt inline as a cold key. The deferred rebuild ran once the command ended.

Since Laravel 12.16, flexible() takes alwaysDefer: true to rebuild after a failed response or job as well. A long-running command is better off with remember() behind a lock.

A rebuild lock with no expiry#

The rebuild lock is taken for 0 seconds unless you pass a number, and on Redis that is a plain SETNX: a key with no TTL. Lock::get() releases it in a finally block, which runs when the closure throws and does not run when the process dies. A worker killed in the middle of a rebuild, or one whose rebuild ran past memory_limit, leaves the lock in Redis for good. The response that triggered it has already gone out with a 200.

From then on no rebuild can take the lock. The stale value is served until the key's lifetime runs out, then the key goes cold and stampedes, and the cycle repeats. With [2, 10] at 20 requests a second, 21 requests ran the closure every time the ten seconds ran out, and nothing was rebuilt in the background again. The fix is a lock that expires:

$report = Cache::flexible(
    'report',
    [300, 900],
    fn () => Report::build(),
    lock: ['seconds' => 30],
);

Give it longer than a rebuild takes, or a second process starts one while the first is still running. With seconds set to five, rebuilds resumed once the dead worker's lock expired.

Clearing a stuck lock#

php artisan cache:clear leaves it where it is. The Redis store keeps locks on its lock_connection, which is the default connection, database 0, while the cache itself is on the cache connection, database 1, and cache:clear only empties that one. To delete one lock, force-release it by name:

Cache::lock('illuminate:cache:flexible:lock:report')->forceRelease();

php artisan cache:clear --locks, new in Laravel 13.24, runs FLUSHDB on the lock connection's database. With the default configuration that is database 0, which is also where the Redis queue keeps its jobs and the Redis session driver keeps sessions. In a fresh Laravel 13.42 app it removed the stuck lock and every queued job with it. Give locks a database of their own before using it:

// config/database.php, under 'redis'
'locks' => [
    'url' => env('REDIS_URL'),
    'host' => env('REDIS_HOST', '127.0.0.1'),
    'username' => env('REDIS_USERNAME'),
    'password' => env('REDIS_PASSWORD'),
    'port' => env('REDIS_PORT', '6379'),
    'database' => env('REDIS_LOCK_DB', '2'),
],

// .env
REDIS_CACHE_LOCK_CONNECTION=locks

With that in place, --locks cleared the lock and the queue kept its jobs.

Seeing a stampede in production#

The usual numbers hide it. Over ten minutes the hit ratio slips from 100% to 99.8%. Redis's keyspace_misses counts a miss for each of those requests whether one process rebuilt the key while the others waited on a lock or every one of them rebuilt it. What tells the two apart is a process writing a key that another process already wrote after both had missed it.

Laravel fires CacheMissed on the miss and KeyWritten on the write, on every store, so a listener can pair them:

// AppServiceProvider::boot()
use Illuminate\Cache\Events\CacheMissed;
use Illuminate\Cache\Events\KeyWritten;
use Illuminate\Support\Facades\Event;
use Illuminate\Support\Facades\Redis;
use Illuminate\Support\Str;

$missed = [];

Event::listen(CacheMissed::class, function (CacheMissed $event) use (&$missed) {
    if (str_starts_with($event->key, 'illuminate:')) {
        return; // Laravel's own keys, such as flexible()'s timestamps
    }

    $missed[$event->key] = true;
    Redis::set("generating:{$event->key}", 1, 'EX', 60, 'NX');
});

Event::listen(KeyWritten::class, function (KeyWritten $event) use (&$missed) {
    if (! isset($missed[$event->key])) {
        return; // a put() with no miss before it, or flexible()'s background rebuild
    }

    unset($missed[$event->key]);

    if (Redis::del("generating:{$event->key}") === 0) {
        Redis::incr('stampedes:'.Str::before($event->key, ':'));
    }
});

The first process to miss a key marks it, and the first to write it clears the mark. A process that missed the key and finds nothing to clear when it writes has rebuilt a value another process had already written. The counter is kept per group of keys, so product:4411 and product:982 both add to stampedes:product. Under the load above it counted 58 of the 59 duplicate builds from remember(), and none for the lock or for flexible() with the key cached. The one it missed began just after the first write had cleared the mark. In Octane or a queue worker the $missed array lives as long as the worker, and keeps every key that was missed and never written.

Sailfish counts the same thing in every process of a Laravel app, for each group of cache keys, beside how long a generation takes and how much time went into generations that ran twice at once. Its Cache page sorts the groups by time spent generating. A cache stampede event goes into the daily digest when a group's keys were generated in two processes at once at least ten times in an hour, each taking 50 ms or more, with the time that wasted and a pointer to Cache::flexible().

The Sailfish page for the product_recommendations cache group over seven days: a 98% hit ratio at 21 lookups a second; 293,499 generations at 840 ms each on average and up to 3.9 s, 68.5 hours in all; 58,593 of them, 20%, begun while another process was generating the same key; and charts of lookups and of generations, with those begun twice at once rising and falling with the day's load.
A demo shop's product_recommendations keys over a week. The hit ratio is 98%, and a fifth of its generations began while another process was already working out the same key.

Frequently asked questions

What is a cache stampede in Laravel?

A cached key expires and every request that reads it before the first one has written it back misses too and runs the Cache::remember() closure itself. With a value that takes a second to build and 50 requests a second, 51 requests built it at once. The hit ratio barely moves, because it happens once per expiry.

Does Cache::flexible() prevent cache stampedes?

Only while the key is still cached. In its stale window flexible() serves the old value and one process rebuilds it after the response. A key that is gone, because the cache was cleared or nobody read it before its second TTL ran out, is built inline by every request that misses it, as with remember(). The second number is the key's whole lifetime from the write, so it can be long, and a lock taken only when the key is missing covers the cold case.

Should I use Cache::flexible() or a lock?

Both, when the value may be a few seconds old: call flexible() when the key is cached, and on a miss call it inside Cache::withoutOverlapping(). A lock alone builds the value once but every request that arrives during the build waits for it; flexible() alone makes no request wait while the key is cached, but stampedes when it is gone. When the value must never be stale, use remember() behind a lock and accept the wait.

How do I stop Cache::remember() from running the closure in many requests at once?

Check the cache, and on a miss run remember() inside a lock: Cache::get($key) ?? Cache::withoutOverlapping("{$key}:lock", fn () => Cache::remember($key, $ttl, $callback), lockFor: 10, waitFor: 5). The first process builds the value and the others wait for the lock, then read it. Give lockFor and waitFor longer than the build takes. withoutOverlapping() is in Laravel 12.47 and later; before that use Cache::lock($name, 10)->block(5, $callback).

Why does Cache::flexible() stop refreshing in the background?

Most often a worker died during a rebuild and left its lock behind: the rebuild lock has no expiry unless you pass lock: ['seconds' => 30]. Until it is removed, the stale value is served until the key expires, and then the key stampedes. The rebuild also does not run after a response with a status of 400 or more, after a failed queued job, or before an artisan command exits.

Does php artisan cache:clear remove cache locks?

No. On Redis, locks live on the store's lock_connection, by default the default connection's database 0, and cache:clear only empties the cache connection's database. cache:clear --locks, in Laravel 13.24 and later, runs FLUSHDB on database 0, which by default also holds the Redis queue's jobs and Redis sessions. Release a single lock with Cache::lock($name)->forceRelease(), or move locks to a database of their own first.