Invalidate a Redis cache without scanning it

Sooner or later every cache needs a "forget everything about this" button. A product changes price, a menu is edited, a plugin updates, and a whole family of cached entries is suddenly wrong. The tempting answer is to find every matching key and delete it. On a small cache that works. On a large one it is slow, it hammers Redis while it runs, and it gets slower as the cache grows.

I measured three ways of clearing a group of keys on a WordPress site I run, then on a synthetic set of 100,000 keys so the numbers were easy to compare. Here is what I found and which one I reach for now.

The three approaches

1. SCAN, then DEL each key

This is what most code does first. SCAN walks the keyspace with a match pattern, and each key it returns gets its own DEL. It never blocks Redis for long, which is why people prefer it to KEYS, but every key costs a network round trip, and the scan itself has to look at every key in the database, not just the ones you want.

2. SCAN, then UNLINK in pipelined batches

Same scan, but instead of one command per key you collect a few hundred keys and send them as a single UNLINK, with several batches in one pipeline. UNLINK removes the keys from the keyspace straight away and frees their memory on a background thread, so large values do not stall the server.

3. A generation counter

Here you do not delete anything. Each group of keys has a small counter, and the counter's value is part of every key name. To invalidate the group you increment the counter. Every old key is now unreachable because nothing will ever ask for that name again.

What it cost on 100,000 keys

The second row is nearly ten times faster than the first for a small change to the code. The third is in a different league, because its cost does not depend on how many keys there are.

Batched UNLINK in PHP

If you need the keys genuinely gone, this is the shape I use with phpredis. It scans in chunks, batches the deletes and sends them in a pipeline:

$redis->setOption(Redis::OPT_SCAN, Redis::SCAN_RETRY);
$it    = null;
$batch = [];

while (($keys = $redis->scan($it, 'shop:menu:*', 1000)) !== false) {
    foreach ($keys as $key) {
        $batch[] = $key;
    }
    if (count($batch) >= 500) {
        $pipe = $redis->multi(Redis::PIPELINE);
        foreach (array_chunk($batch, 250) as $chunk) {
            $pipe->unlink($chunk);
        }
        $pipe->exec();
        $batch = [];
    }
}
if ($batch) {
    $redis->unlink($batch);
}

Two checks before you ship it. First, confirm your Redis is 4.0 or later, since UNLINK did not exist before that. Second, if you run Redis Cluster, a multi-key command only works when every key in it hashes to the same slot. Either group keys with a hash tag such as {menu} in the name, or fall back to one UNLINK per key inside the pipeline, which is still far cheaper than separate round trips.

The one-INCR pattern

For anything that gets cleared often, I now use a generation counter. Reading and writing look like this:

function menu_key(Redis $redis, string $id): string {
    $gen = $redis->get('gen:menu') ?: 0;
    return "menu:{$gen}:{$id}";
}

// Write: always with a TTL.
$redis->set(menu_key($redis, $id), $html, ['ex' => 3600]);

// Read.
$html = $redis->get(menu_key($redis, $id));

// Invalidate the whole group.
$redis->incr('gen:menu');

The counter read is one extra GET per request, and you can hold it in a PHP variable for the rest of the request so it is only fetched once. Invalidation is a single command no matter how big the group is.

The trade-offs

None of these is free, and the right choice depends on what matters more to you: memory or speed.

• Every key needs a TTL with a generation counter. Old generations are never deleted, only abandoned. Without an expiry they sit in memory until Redis evicts them, and if your eviction policy is noeviction they will sit there until Redis refuses writes. I check with redis-cli --scan --pattern 'menu:*' | head and TTL on a few results.
• Dead keys take a while to leave. Straight after an increment, memory use does not drop. It falls gradually as the old keys expire. If you are short of memory, that lag matters, and batched UNLINK is the better tool.
• UNLINK frees memory asynchronously. The keys vanish immediately, but used_memory in INFO memory may take a moment to fall. Do not read that as a failed delete.
• Cluster slots apply to multi-key commands. As above, a batched UNLINK across slots fails with a CROSSSLOT error. The generation counter has no such problem, since it touches one key.
• A lost counter is a full flush. If the counter key itself is evicted, it resets to zero and old generation zero entries may come back to life. Give the counter no TTL, and if you use an allkeys eviction policy, consider starting it from a timestamp rather than zero.

How to check yours

To see whether invalidation is hurting you now, run redis-cli --latency in one terminal while you trigger a cache clear in another. If latency jumps while the clear runs, look at the code that does it. A loop with one DEL per key is the usual culprit. INFO commandstats before and after the clear will also show you how many commands it really sent.

My rule of thumb now: groups that change often or can grow large get a generation counter and a TTL on every key. One-off clean-ups, and anything where memory is tight, get pipelined UNLINK. Per-key DEL in a loop does not get used at all.

Further reading: the Redis documentation for UNLINK and SCAN.