What is a service mesh, and what does it give you that you'd otherwise have to build into each application?

technical-conceptual · Junior level · cloud-devops-security

What the interviewer is really asking

Assess whether the candidate understands a service mesh as an infrastructure layer that handles service-to-service concerns — mTLS, retries, traffic management, observability — transparently to application code, and knows the sidecar versus newer sidecarless (ambient) data-plane models at a high level.

What to say

• Define it as a dedicated infrastructure layer for service-to-service communication: it intercepts traffic between your services and handles cross-cutting concerns — mutual TLS, retries, timeouts, traffic splitting, and telemetry — without the application code having to implement them.
• Explain the data plane: traditionally a sidecar proxy (like Envoy) runs next to each service and transparently captures its traffic; newer 'ambient'/sidecarless models (e.g. Istio ambient) move the L4 work to a per-node proxy to cut the sidecar overhead.
• Stress the payoff: you get consistent encryption-in-transit, observability, and traffic policy across every service in one place, instead of each team re-implementing retry logic and TLS in every language and codebase.

What to avoid

• Confusing a service mesh with a load balancer or an API gateway — a mesh is about internal service-to-service traffic and policy, not just routing inbound requests.
• Claiming you always need a mesh — for a handful of services the operational complexity often outweighs the benefit; it pays off as the number of services and the need for uniform mTLS/observability grows.
• Saying the application has to be rewritten to use a mesh — the point is that the proxy handles it transparently, so app code mostly doesn't change.

Example answers

Strong: A service mesh puts a proxy in the path of every service's traffic so things like mutual TLS and retries happen at the infrastructure layer, not in each app. On a project we got automatic mTLS between all services just by adding the mesh — no team had to write TLS code in their service, and we got per-service latency and error metrics for free.

Weak: A service mesh is the load balancer that sends user requests to your backend services.