Your team is deciding whether to keep running your service on the JVM or move to an ahead-of-time compiled native image to cut cold-start latency. How would you reason about that trade-off?

technical-conceptual · Senior level · software-engineering

What the interviewer is really asking

Assesses whether the candidate understands the JIT-versus-AOT trade-off in terms of workload shape — adaptive peak throughput from a warmed runtime versus immediate startup from AOT — rather than treating 'native is faster' as a blanket truth.

What to say

• Make it a workload question: a JIT warms up by profiling hot methods and recompiling them with tiered compilation, so it pays a warmup tax but reaches higher peak throughput on long-lived processes.
• Explain what AOT buys and costs: GraalVM Native Image starts in milliseconds with lower memory, ideal for serverless and scale-to-zero, but gives up the JIT's runtime profiling so steady-state throughput on a hot path can be lower.
• Decide from the traffic pattern: bursty or short-lived invocations favor AOT; continuously-running, throughput-sensitive services that stay warm often still favor the JVM — and measure both on your real workload before committing.

What to avoid

• Asserting native images are simply faster, ignoring that they trade away adaptive peak optimization.
• Ignoring the build-time and tooling cost of native compilation, including closed-world assumptions that break reflection-heavy code.
• Choosing based on a microbenchmark instead of the service's real request distribution and how long instances actually live.

Example answers

Strong: For our scale-to-zero functions the JVM warmup tax was the whole problem — first requests after a cold start were seconds slower. I moved those to a GraalVM native image and p99 cold-start dropped from ~4s to under 200ms. But for the always-on ingestion service, I kept the JVM, because it stays warm for days and the JIT's profile-guided recompilation beat the native image on sustained throughput in our load test.

Weak: Native images are compiled ahead of time so they're faster than the JVM, so I'd move everything to GraalVM to get better performance and lower memory.