A third-party partner integrates with our API and occasionally hammers it with bursts that degrade service for everyone else. We don't own their client. How would you protect the API?

system-design · Mid level · software-engineering

What the interviewer is really asking

Assesses server-side throttling and graceful-degradation design from the provider's side — per-tenant quotas, fair isolation so one noisy client can't starve others, and the correct 429 contract — distinct from implementing a distributed counter.

What to say

• Enforce per-client (per-API-key/tenant) limits at the gateway, not a single global limit: the goal is fairness isolation so one partner's burst can't consume the capacity the others depend on — give each key its own quota and ideally a separate token bucket.
• Return the correct contract so a well-behaved client can adapt: respond 429 Too Many Requests with a Retry-After header and X-RateLimit-Limit/Remaining/Reset, and allow a bounded burst (token bucket) so legitimate spikes aren't punished while sustained abuse is shed.
• Add defense for the client you don't control: since they may ignore Retry-After, back the per-key limit with a hard ceiling and consider load-shedding or a concurrency cap; pair it with a separate timeout/circuit on your side so their burst can't tie up your workers, and alert so you can talk to the partner.

What to avoid

• Applying only a single global rate limit, which a hot partner can saturate, starving every other client even though none of them is over their own limit.
• Returning a generic 500 or silently dropping requests under load instead of a 429 with Retry-After, so the client has no signal to back off.
• Assuming the partner will respect Retry-After and stop, with no hard ceiling, concurrency cap, or load-shedding for a client that just keeps retrying.

Example answers

Strong: Since I can't change their client, I push the protection entirely server-side and make it per-key. Each partner's API key gets its own token-bucket quota at the gateway, so a burst from one consumes only their own budget — the others are isolated and keep their capacity. When they exceed it I return 429 with Retry-After and the X-RateLimit headers so a sane client throttles itself. Because this partner might ignore that, I add a hard concurrency cap per key and load-shed beyond it, and I put a timeout on those requests so they can't pin my workers. I'd also alert on sustained 429s for that key so we can reach out and fix it at the relationship level.

Weak: I'd add a global rate limit on the API so total traffic can't exceed what the servers handle. Once we hit the cap, extra requests get rejected.