A cache-or-load helper. Looks correct. On cold boot or after the cache TTL fires, every concurrent request misses, every miss runs the expensive query. The database load spikes; cascading failures follow.

The vulnerable diff

tscache/get-or-load.ts

// cache/get-or-load.ts
import { redis } from "./redis";

export async function getOrLoad(key: string) {
  // BUG: concurrent misses each run the query
  let cached = await redis.get(key);
  if (cached) return JSON.parse(cached);

  const fresh = await expensiveQuery(key); // 800ms, 5 SQL joins
  await redis.set(key, JSON.stringify(fresh), "EX", 60);
  return fresh;
}

What is wrong

Cache stampede (also: dogpile, thundering herd) is the worst-case failure mode of a naive cache. Under steady load, the cache absorbs 99% of reads. On cold start, on TTL expiry, on cache eviction, every concurrent request misses simultaneously, every miss runs the underlying query. Single-instance services see this as a brief latency spike. Multi-instance services see it as a thundering herd against the database — 1000 instances × N concurrent requests each = N×1000 queries against the same key. Production databases routinely fall over under this pattern.

The attack

Reproducer: 500 concurrent requests against an empty cache.

textexploit

# Cold cache, 500 parallel reqs against /api/popular-page
# Each request misses, runs expensiveQuery (800ms, 5 joins).
# Database CPU: 100%. Latency: p99 = 12 seconds. Some requests time out.
# After 1 second the cache is populated. Subsequent reqs serve in 2ms.
# The damage is the 1-second window — and on a busy site, that window
# happens at every TTL boundary.

Symptoms in production: periodic latency spikes correlated with cache TTL; database CPU saw-tooths; SREs add a CDN/cache layer in front to mask without fixing root cause.

Mesrai's review comment

textreview-comment.txt

mesraipilot · Bot · reviewed 90 sec ago

[mesrai] [code-review] [Performance] [Stampede] [high]

Cache stampede. On cold cache or TTL expiry, every concurrent
request runs the expensive query. Database overloads, latency
spikes, sometimes cascades to outage.

Two layers of fix:

  1. Single-flight within a process: only one in-flight load per key.
     If a second request arrives while the first is loading, it
     waits for and shares the same Promise.

  2. Distributed lock for multi-instance: a Redis lock so only one
     instance loads at a time across the fleet. Or use a stampede-
     safe cache library (e.g. cache-manager with the 'lock' option).

  import pMemoize from "p-memoize";

  const singleflight = pMemoize(
    async (k: string) => expensiveQuery(k),
    { cacheKey: ([k]) => k, maxAge: 60_000 }
  );

  export async function getOrLoad(key: string) {
    const cached = await redis.get(key);
    if (cached) return JSON.parse(cached);

    const fresh = await singleflight(key);
    await redis.set(key, JSON.stringify(fresh), "EX", 60);
    return fresh;
  }

Also stagger TTLs (TTL + small jitter) so different keys do not
expire at the same instant.

The fix

tscache/get-or-load.ts (fixed)

// cache/get-or-load.ts — fixed
import { redis } from "./redis";
import pMemoize from "p-memoize";

const loadOne = pMemoize(
  async (key: string) => expensiveQuery(key),
  { cacheKey: ([k]) => k, maxAge: 60_000 }
);

function ttlWithJitter(seconds: number) {
  return seconds + Math.floor(Math.random() * Math.min(seconds * 0.1, 10));
}

export async function getOrLoad(key: string) {
  const cached = await redis.get(key);
  if (cached) return JSON.parse(cached);

  const fresh = await loadOne(key); // single-flight per process
  await redis.set(key, JSON.stringify(fresh), "EX", ttlWithJitter(60));
  return fresh;
}

Two changes. `p-memoize` ensures only one in-process query per key — subsequent callers await the same Promise. TTL jitter prevents the next stampede by spreading expiry across a small window. For multi-instance protection, add a Redis lock (`SET key NX EX 30`) and have other instances wait briefly when the lock is taken.

Why human review missed it

Stampede is invisible until you have enough traffic to trigger it. Local dev hits warm caches; staging is single-instance; production is the first place the fan-out happens. The bug also looks correct in isolation — there is a cache check, there is a load, there is a write-back. The race is between concurrent calls of the same function. Mesrai catches it by flagging every get-then-load pattern without a single-flight wrapper.

Related rules + further reading

Mesrai rule pack: performance/no-cache-stampede — flags get-then-load without single-flight or distributed lock.

Theo Schlossnagle and the Heroku post-mortems library both have classic essays on cache stampede.

Causes ~10% of cache-related incidents in our review data.

Takeaway

Single-flight in-process. Distributed lock across instances. TTL jitter. Three changes that cover every cache-stampede shape.

Cache Stampede on Cold Boot: A Mesrai-Caught Architectural Smell