A Redis cache layer that does not go stale

A cache in front of the catalogue took the product page from 800ms to 40ms. It also meant a price change took up to fifteen minutes to appear, and for those fifteen minutes the site was quoting a number the database no longer agreed with. Fast and wrong is not obviously better than slow and right.

The symptom

Support raised it as a bug report about pricing: a customer had been shown one price on the listing and a different one in the cart. Both were real prices, from different points in time, sitting in different caches with different expiry times.

$ redis-cli TTL product:5512
(integer) 641

$ redis-cli TTL catalogue:brand:17:page:1
(integer) 209

Two entries containing the same price, expiring eleven minutes apart. Nothing was going to reconcile them except time.

Why it happens

A TTL is a statement about how long you are willing to be wrong, not about when the data changed. It is the correct tool when the underlying data has no change event you can observe — an external rate, an aggregate that is expensive to recompute — and the wrong one when the application itself performs the write.

In this case the application knew precisely when the price changed. It simply was not telling the cache.

The fix

Key design first

Invalidation is only tractable if the keys can be reasoned about. That means a consistent scheme, decided once, where the key names the thing it holds and every input that varies it.

// entity: one key per record
"product:{$id}"

// derived view: name every input that varies it
"catalogue:brand:{$brandId}:sort:{$sort}:page:{$page}"

// per-customer: scope explicitly, never share by accident
"cart:{$customerId}"

Invalidate on write

With entity keys, the write path can delete exactly what it invalidated. This is the whole change: the cache stops guessing and starts being told.

final class CachedProducts implements Products
{
    private $inner;
    private $redis;

    public function __construct(Products $inner, Redis $redis)
    {
        $this->inner = $inner;
        $this->redis = $redis;
    }

    public function find($id)
    {
        $key    = "product:{$id}";
        $cached = $this->redis->get($key);

        if ($cached !== false) {
            return unserialize($cached);
        }

        $product = $this->inner->find($id);

        if ($product !== null) {
            $this->redis->setex($key, 3600, serialize($product));
        }

        return $product;
    }

    public function save(Product $product)
    {
        $this->inner->save($product);
        $this->redis->del("product:{$product->id()}");
    }
}

A decorator rather than a flag inside the repository, so the uncached implementation stays free of cache concerns and the tests for it do not need Redis. The TTL stays as a backstop — a safety net for the invalidation you forgot, not the mechanism.

Two details in that code are worth stating explicitly. The first is === false rather than a truthiness check: a legitimately cached empty string or integer zero is falsy, and a loose test turns every one of those into a cache miss that re-queries the database forever.

The second is serialize() rather than json_encode(). JSON is smaller and readable from redis-cli, which is genuinely useful when debugging, but it loses the class — a Product comes back as a plain array, and every caller has to rehydrate it. Serialising keeps the object at the cost of coupling the cache to the class definition, so a deploy that changes the class must invalidate the keys. Neither is wrong; the choice just has to be made deliberately rather than by whichever example was copied.

The derived views are the hard part

Deleting product:5512 is easy. The listing pages that included product 5512 are the problem: there may be forty of them, across sort orders and pagination, and the key does not say which products it contains.

The instinct is to scan for them, and that instinct is what takes the site down.

// never in production: Redis is single-threaded, and this
// blocks every other client until it finishes
foreach ($redis->keys('catalogue:brand:17:*') as $key) {
    $redis->del($key);
}

The better answer avoids scanning altogether: keep a version number per brand, and include it in the key. Bumping the version orphans every old key at once, and they expire on their own.

$version = $this->redis->get("brand:{$brandId}:v") ?: 1;
$key     = "catalogue:brand:{$brandId}:v{$version}:sort:{$sort}:page:{$page}";

// a product in this brand changed:
$this->redis->incr("brand:{$brandId}:v");   // every old key is now unreachable

One INCR invalidates an unbounded number of derived keys in constant time. The orphans occupy memory until their TTL expires, which is the price, and it is why maxmemory-policy must be set to an allkeys-lru variant rather than left at noeviction.

Verifying it worked

Two numbers, neither of which is the hit rate on its own. The hit rate says the cache is being used; the write-to-visible time says it is being used correctly.

$ redis-cli INFO stats | grep keyspace
keyspace_hits:1841266
keyspace_misses:97431
# 95.0% hit rate

$ time ./bin/price-change-visible-check
changed at 14:22:07.114, visible at 14:22:07.147
real    0m0.041s

Fifteen minutes to thirty-three milliseconds. The hit rate barely moved, which is the point — invalidating correctly did not cost throughput, because a price change is rare compared to a page view.

What this costs

Every write path now carries a cache responsibility, and the compiler will not remind anyone. A new admin screen that updates products through a different code path will produce exactly the stale-price bug this was built to fix, and it will be reported the same way. Routing all writes through the decorated repository is the only structural defence, and it requires the discipline to keep doing so.

There is also a new dependency in the request path. Redis going away must degrade to slow rather than to broken, which means catching connection failures around every cache read and falling through to the database — and testing that path, because it is the one nobody exercises until the afternoon it matters.