A popular cache key expires and your database suddenly gets hammered by a flood of concurrent requests all trying to recompute the same value. What is this, and how do you prevent it?
technical-conceptual · Mid level · software-engineering
What the interviewer is really asking
Assesses whether the candidate recognizes the cache stampede / thundering herd problem and can compare concrete mitigations (locking/single-flight, probabilistic early refresh, TTL jitter) with their trade-offs.
What to say
- Name the problem: cache stampede (thundering herd / dog-pile) - when a hot key expires, many concurrent requests miss simultaneously and all recompute against the origin, spiking load.
- Offer mitigations and when each fits: request coalescing / single-flight or a short-lived lock so only one request recomputes while others wait or serve stale; probabilistic early expiration so one request refreshes just before TTL; and TTL jitter so keys don't all expire at the same instant.
- Mention serving stale-while-revalidate as a UX safety valve and that you'd verify with cache hit-rate and origin QPS metrics.
What to avoid
- Just shortening or lengthening the TTL without addressing the synchronized-expiry burst that causes the stampede.
- Claiming a bigger cache or more app servers solves it, which doesn't stop every instance from missing the same key at once.
- Proposing a lock without thinking about lock TTL/deadlock or what the waiting requests do (block forever vs serve stale).
Example answers
Strong: That's a cache stampede: a hot key expires and every concurrent request misses and recomputes at once. I'd coalesce them with a single-flight lock (Redis SET NX with a short TTL) so one request recomputes while the rest serve the stale value or briefly wait, add TTL jitter so keys don't expire in lockstep, and consider probabilistic early refresh so it's renewed just before expiry. I'd confirm the fix on origin QPS and hit-rate dashboards.
Weak: The cache entry expired so everyone hit the DB. I'd set a much longer TTL so it expires less often, and if that's not enough we can add a bigger Redis instance to take more load off the database.