If someone reports your API 'feels slow,' why is the average response time often a misleading number, and what would you look at instead?

technical-conceptual · Junior level · software-engineering

What the interviewer is really asking

Assesses whether the candidate understands that averages hide tail latency and can reach for percentiles (p95/p99) as the metric that reflects what slow users actually experience — a step beyond knowing 'measure latency.'

What to say

• Explain why the average lies: a few very fast responses can pull the mean down and hide a slow tail, so an average that looks fine can still mean a meaningful slice of requests are painfully slow.
• Reach for percentiles: look at p50 (the typical request), p95, and p99 — p99 being the latency that 99% of requests come in under, which captures what your slowest 1% actually feel. Tracking the distribution, not one number, shows the real shape.
• Add why the tail matters: at scale, that slow 1% is a lot of real people, and in a chain of services each hop's tail compounds, so a single slow dependency can make many end-to-end requests slow. I'd also distinguish latency from throughput — 'slow' is usually a latency problem.

What to avoid

• Insist the average is enough, or report only one aggregate number without acknowledging it hides the distribution.
• Confuse latency (how long one request takes) with throughput (how many per second) — 'feels slow' is almost always latency.
• Jump straight to 'add more servers' before measuring where the time goes; more capacity won't fix a slow query in the request path.

Example answers

Strong: The average is misleading because it can be dragged down by lots of fast responses and hide a slow tail — if most requests are 50ms but 5% take two seconds, the mean still looks okay while a real chunk of users are suffering. So I'd look at percentiles: p50 for the typical request, and p95/p99 for the tail — p99 is the latency 99% of requests come in under, so it tells you what your slowest users actually experience. Watching the whole distribution shows whether the app is genuinely fast or just fast on average. The tail matters because at scale 1% is a lot of people, and across multiple services each hop's slow tail adds up, so one slow dependency drags the whole request. I'd also keep latency and throughput separate — 'feels slow' is a latency question.

Weak: I'd check the average response time, and if it's high I'd add more servers so requests get handled faster. As long as the average is low the API isn't really slow, so the report is probably just one user's connection.