Why Our Laravel Response Cache Kept Missing: Click IDs, Cache Keys and a Cached noindex

Our search pages sat behind CacheResponse::for(3600), and a rendered page was supposed to be served from cache for an hour. Laravel Nightwatch started alerting on routes slower than 1,000ms, and those pages kept tripping it at 1.5 to 2.1 seconds. Every slow sample was a response-cache miss. With a one-hour lifetime and steady traffic, misses should have been rare, and they weren't.

This post goes through the causes we found in a production Laravel app using spatie/laravel-responsecache 8.4, Spatie's open-source full-page cache for Laravel. Click ids split the cache into one entry per visitor, a second cache inside the page had keys that never matched, and one cause wasn't a hit-rate problem at all: a page rendered for a visitor from an ad was cached with a noindex tag and served to Googlebot. Route and class names are changed.

How responsecache builds a key#

The package's DefaultHasher hashes the host, the path with its normalised query string, the HTTP method and a suffix from the cache profile. The default suffix is the authenticated user's id, or an empty string for guests. Parameters listed in ignored_query_parameters are removed from the query string first, so /destinations/croatia?utm_source=newsletter and /destinations/croatia hash to the same key.

Our app extends the hasher and appends the currency, the locale and the user:

class AppHasher extends DefaultHasher
{
    public function getHashFor(Request $request): string
    {
        return parent::getHashFor($request)
            . '-' . Currency::getCurrent()
            . '-' . App::getLocale()
            . '-' . (auth()->check() ? auth()->id() : 'guest');
    }
}

That is a sensible key for a page that shows prices, but it multiplies the entries each URL needs. One destination page in eight locales and four currencies is 32 guest entries, each filled by its own first visitor. Logged-in users get their own entries on top of that. The hit rate that matters is the guests', and every one of those entries has to survive long enough to be hit.

Click ids that split the cache#

The package ships with the five utm_* parameters, gclid and fbclid in ignored_query_parameters. That list is older than most of the click ids arriving today. Google adds srsltid to organic results for sites with Merchant Center auto-tagging, and Google Ads now sends gad_source, gad_campaignid, gbraid and wbraid alongside or instead of gclid. Every one of these carries a unique value per click, so every visitor who arrived with one got a fresh cache entry that nobody else would ever hit, and a full render to go with it.

None of them is read on the server, so all of them can go in the list:

'ignored_query_parameters' => [
    'utm_source', 'utm_medium', 'utm_campaign', 'utm_term', 'utm_content', 'utm_id',
    'gclid', 'gad_source', 'gad_campaignid', 'gbraid', 'wbraid', 'dclid', 'srsltid',
    '_gl',          // Google Analytics cross-domain linker
    'fbclid', 'igshid',
    'msclkid',      // Microsoft Ads
    'ttclid', 'twclid', 'li_fat_id', 'yclid',
    'mc_cid', 'mc_eid',   // Mailchimp
    '_hsenc', '_hsmi',    // HubSpot
],

Before adding a parameter, search the codebase for it. If a controller or a view reads it, for example to attribute a booking to a campaign on the server, ignoring it in the key means one visitor's value is cached and shown to the next. Attribution that happens in the browser, through an analytics script reading location.search, is unaffected.

Ignored parameters still change the page#

Adding to the list created a problem that had in fact been there since day one for utm_*. The hasher ignores the parameter when building the key, but the controller still sees it. Our listing controllers mark a page noindex when its query string contains anything other than page, a common rule for keeping filter permutations out of the index. The paginator's links also carried the query string forward.

So when the first visitor after an expiry arrived at /destinations/croatia?utm_source=newsletter, the page rendered with a noindex robots tag and with utm_source=newsletter in every pagination link, and it was stored under the same key as /destinations/croatia. For the next hour every visitor and every crawler got that copy. With srsltid in the list, the first visitor from an organic Google result would have done the same thing, far more often.

The way out is to let tracked requests read the clean URL's entry but never write one. That works because of the order in which the CacheResponse middleware does things. It looks the request up before calling the controller, using only the hasher. Only after the response comes back does it ask the cache profile whether to store it. A profile can therefore refuse to store tracked requests without affecting whether they're served:

namespace App\ResponseCache;

use Illuminate\Http\Request;
use Spatie\ResponseCache\CacheProfiles\CacheAllSuccessfulGetRequests;

class SkipTrackedRequestsCacheProfile extends CacheAllSuccessfulGetRequests
{
    public function shouldCacheRequest(Request $request): bool
    {
        return parent::shouldCacheRequest($request) && ! $this->hasTrackingParameters($request);
    }

    private function hasTrackingParameters(Request $request): bool
    {
        return collect($request->query->keys())
            ->intersect(config('responsecache.ignored_query_parameters', []))
            ->isNotEmpty();
    }
}

'cache_profile' => App\ResponseCache\SkipTrackedRequestsCacheProfile::class,

A visitor from an ad now gets the cached clean page if one exists, and a fresh render with their own noindex and links if it doesn't. Only a visit to the clean URL fills the cache. The cost is that a page reached mostly through tagged links fills a little more slowly, which is a fair trade for never serving one visitor's page to everyone else.

The cache inside the cached page#

A response-cache hit skips the whole request, but the listing on these pages is a Livewire component. Its first render happens inside the cached page, and every later update, a filter change or the next page, is a POST to Livewire's update endpoint. The default cache profile only stores GET requests, so none of those updates is ever response-cached. That's why the list query has its own cache:

$cacheKey = 'listings_' . $filters->hash() . '_' . http_build_query($extras);

return Cache::remember($cacheKey, 300, fn () => $this->listingService->search($request, $filters));

On a response-cache miss, the page runs that query twice: once for the JSON-LD block that describes the results to search engines, and once for the Livewire list. Both callers pass the same filters, so the second should have been a hit. It never was. $filters->hash() defaulted one filter, near_locations, to true when the request didn't set it, and the Livewire component always sent it explicitly as 1. The filter array is JSON-encoded before it's hashed, and true and 1 don't encode to the same string.

// Cast to match how the filter consumes it, so true / 1 / "1" share one entry
$filters['near_locations'] = (bool) ($this->requestQuery('near_locations') ?? true);

The filter itself had always treated all three as the same value, so the results were identical and nothing looked wrong. The slowest query on the page simply ran twice on every miss. Any cache key built from request input has this weakness: "1" from a query string, 1 from JSON and true from a PHP default are the same thing to the code and three things to a hash. Cast each value to the type the code consumes before it goes into the key.

Clearing less than everything#

Every entry that gets thrown away has to be rendered again by its next visitor, so how the cache is invalidated matters as much as how it's keyed. ResponseCache::clear() with no arguments is the obvious tool, and it removes more than its name suggests.

responsecache 8 offers two narrower tools. Neither drops in cleanly with a key like ours, and the reasons are worth knowing before you reach for them.

Tags. CacheResponse::for(3600, ['vendor:42']) tags the stored response, and ResponseCache::clear(['vendor:42']) flushes only that tag. Tags need a store that supports them, such as Redis or Memcached. On the file store, the default, they throw.

Forgetting URLs. ResponseCache::forget('/vendors/acme') builds a request for the URL, hashes it and deletes that key. With the default hasher that's the guest entry for the page. With a hasher like ours, the key also includes the locale, the currency and the user of the request doing the forgetting. Called from the admin's request, it removes the admin's own variant of the page and leaves the 32 guest variants in place. selectCachedItems() can set headers, cookies and the profile's suffix on the built request, but our hasher reads the locale and currency from application state, so forgetting every variant means looping over them and setting that state yourself.

Counting hits, misses and clears#

A low hit rate is easier to explain once you can see it per route, and a clear is easier to trace when it's logged with the request that caused it. The package fires an event for each of these, so it's a few lines in a service provider:

use Spatie\ResponseCache\Events\CacheMissedEvent;
use Spatie\ResponseCache\Events\ClearingResponseCacheEvent;
use Spatie\ResponseCache\Events\ResponseCacheHitEvent;

Event::listen(ClearingResponseCacheEvent::class, fn () => Log::info('Response cache cleared', [
    'url' => app()->runningInConsole() ? 'console' : request()->fullUrl(),
    'user' => auth()->id(),
]));

Event::listen(ResponseCacheHitEvent::class, fn ($e) => Cache::increment('rc:hit:'.$e->request->route()?->getName()));
Event::listen(CacheMissedEvent::class, fn ($e) => Cache::increment('rc:miss:'.$e->request->route()?->getName()));

A clear logged with a URL points straight at the code path that triggered it. Two details about the counters: the miss event fires for every request that reaches the controller, including tracked requests the profile won't store, and X-Cache-Status response headers only appear with APP_DEBUG on, so in production the events are the thing to watch.

Testing it through the real middleware#

Mocking the profile would only have tested our own condition. The tests that caught the ordering assumption register a throwaway route behind the real middleware with an array store and count renders:

config(['responsecache.enabled' => true, 'responsecache.cache.store' => 'array']);

Route::middleware(CacheResponse::for(3600))->get('/probe', function () {
    $this->renders++;

    return response('rendered '.$this->renders, 200, ['Content-Type' => 'text/html']);
});

$this->get('/probe?srsltid=abc&gclid=def')->assertSee('rendered 1');
$this->get('/probe')->assertSee('rendered 2');   // the tracked visit stored nothing
$this->get('/probe?srsltid=xyz')->assertSee('rendered 2');   // but it reads the clean entry

The package's profiles already treat the testing environment as not running in the console, so responses do get cached in feature tests without any extra setup.

The mismatched list key is the same bug that shows up in queued work, where two dispatches that mean the same thing get different unique ids and both run. Laravel's job uniqueness controls cover how to keep the same work from being queued twice.