Users of your search feature complain that a query they ran twice returns slightly different result counts, and that a product they just edited took a few seconds to show up. How do you explain these behaviors and decide which ones are acceptable to leave as-is?

system-design · Staff-principal level · software-engineering

What the interviewer is really asking

Probes operational depth on a distributed search cluster — why refresh-interval lag and per-shard scoring/bouncing results produce these symptoms, and the judgment to distinguish a benign trade-off (acceptable) from a real defect (must fix), rather than treating every inconsistency as a bug.

What to say

• Explain the freshness lag first: an edited document isn't searchable until the next refresh exposes a new segment (the default ~1s near-real-time interval, plus indexing-pipeline lag), so a few-seconds delay after an edit is the expected, tunable cost of indexing throughput — not a bug, and you'd confirm it matches the configured refresh interval before 'fixing' anything.
• Diagnose the bouncing counts: in a sharded index, term/document statistics used for relevance scoring are per-shard, so the same query can score and order documents slightly differently depending on which shard replicas answer — and approximate aggregations and a shifting set of in-flight segments make counts wobble; this is a known distributed-scoring artifact, addressable with techniques like a consistent routing/preference or distributed term-statistics if exactness is required.
• Apply judgment on what to fix: leave the benign trade-offs (a 1-2s indexing delay, tiny count jitter on a relevance-ranked result set where exact totals don't matter) and document them as expected; treat as defects the cases where it actually breaks the product — a user not seeing their OWN just-saved edit (fix with read-your-writes routing or a forced refresh on that doc), or a wildly wrong total where the number is load-bearing (pin scoring/use exact counts).

What to avoid

• Treat every observed inconsistency as a bug and chase perfect, instantaneous, globally-identical results, paying large throughput and latency costs to eliminate jitter that doesn't affect the user experience.
• Dismiss all of it as 'eventual consistency, nothing to do', when the user-not-seeing-their-own-edit case genuinely is a defect worth fixing.
• Misattribute the symptoms — e.g. blame caching or a code bug — without naming the refresh interval and per-shard scoring as the actual mechanisms.

Example answers

Strong: Two different mechanisms are at play. The few-seconds delay after an edit is refresh lag: a document only becomes searchable when the next refresh exposes a new segment, about a second by default plus pipeline lag, so that delay is the expected cost of indexing throughput — I'd confirm it lines up with the configured refresh interval before treating it as anything. The bouncing counts come from sharding: relevance statistics and counts are computed per shard, so which replica answers can slightly change scoring and totals, and approximate aggregations wobble. That's a known distributed-scoring artifact, not a bug in our code.

Weak: It's an eventual consistency system, so some inconsistency is just expected and there's nothing really to do about it.