Under load, your service occasionally produces a wrong result — a counter that's off, a duplicate record — but it never reproduces in a single-threaded test. How do you reason about what's going wrong and fix it?

technical-conceptual · Mid level · software-engineering

What the interviewer is really asking

Assesses whether the candidate can recognize a race condition over shared mutable state, identify the non-atomic read-modify-write at its core, and choose an appropriate fix (atomic operation, narrow critical section, or eliminating shared state) rather than guessing.

What to say

• Recognize the signature: intermittent, load-dependent, won't reproduce single-threaded — that's a race condition, where two concurrent executions touch shared mutable state and correctness depends on the accidental interleaving of their operations.
• Find the non-atomic step: the usual culprit is a read-modify-write that isn't atomic — read the counter, add one, write it back — so two threads both read the same value and one increment is lost; or a check-then-act (look up, then insert) where both pass the check and both insert.
• Pick the fix to the situation: make the operation atomic (a database-side increment, an atomic compare-and-swap, or a unique constraint that turns the duplicate into a caught conflict), narrow a critical section with a lock around just the shared access, or best of all eliminate the shared mutable state — confine it per-request or make it immutable so there's nothing to race on.

What to avoid

• Calling it 'flaky' or blaming the framework and adding retries, instead of identifying the shared state and the non-atomic operation that actually causes the corruption.
• Wrapping huge swaths of code in one coarse lock to make the symptom disappear, trading correctness debugging for a throughput bottleneck and possible deadlocks.
• Assuming a higher-level language or an async runtime makes your code automatically thread-safe, when shared mutable state races regardless of language.

Example answers

Strong: Intermittent, only under load, never single-threaded — that's a race condition over shared mutable state. The off-by-some counter smells like a non-atomic read-modify-write: two requests both read the same value, both add one, and one increment is lost. The duplicate record smells like check-then-act — both look up, both see 'not there', both insert. I'd fix the counter with an atomic database increment instead of read-modify-write in app code, and the duplicate with a unique constraint so the second insert fails cleanly and I handle the conflict. Where I can, I'd just remove the shared state — keep it per-request or immutable — so there's nothing to race on rather than reaching for a lock.

Weak: If it only happens sometimes under load it's probably a flaky timing thing in the framework or the database. I'd add some retries so it self-corrects, and maybe wrap the whole request handler in a synchronized block or a lock so only one thing runs at a time. We're on a modern async runtime so it should mostly be thread-safe anyway.