You're designing the topic layout for a high-volume Kafka stream and the team wants both strict ordering and high throughput. Walk me through how you'd choose the number of partitions and the partition key, and the trade-off you're making.

technical-conceptual · Senior level · data-ml

What the interviewer is really asking

Assesses whether the candidate understands that Kafka ordering is per-partition (so the key choice and partition count directly trade ordering scope against parallelism), plus consumer-group and rebalance implications, rather than treating partition count as a pure throughput dial.

What to say

• State the core guarantee: Kafka only orders messages within a single partition, and a key hashes to one partition — so all events for a given key are ordered, but there's no global order across partitions. That makes the key the unit of ordering, not a detail.
• Tie partition count to both throughput and ordering: more partitions buys more consumer parallelism (a consumer group can have at most one active consumer per partition), but it widens the no-ordering-across-partitions surface and adds overhead — so pick the key at the grain you actually need ordered (e.g. per user or per account) and size partitions for peak throughput and headroom, not arbitrarily high.
• Account for skew and operations: a hot key concentrates load on one partition and caps your parallelism there, so check key distribution; and use the cooperative-sticky/incremental rebalance protocol (the default since Kafka 2.4+/3.0, with KIP-848 the next generation) so adding consumers doesn't stop-the-world reprocess everything.

What to avoid

• Treating partition count as a pure throughput knob and ignoring that ordering is only guaranteed within a partition, so a bad key silently breaks per-entity order.
• Choosing a random or null key for even spread when the use case actually requires per-entity ordering, or a single hot key that bottlenecks one partition.
• Setting an enormous partition count 'to be safe' without weighing the per-partition overhead, rebalance cost, and the fact that consumers in a group are capped at one per partition.

Example answers

Strong: The first thing I'd anchor on is that Kafka only guarantees order within a partition, and the key hashes to a partition — so the key is the unit of ordering. If we need per-user order, I key by user_id and every user's events stay ordered, but there's no global order across users, which is usually fine. Partition count then trades throughput against that: more partitions means more parallelism, since a consumer group can run at most one consumer per partition, but it widens the cross-partition no-order surface and adds overhead, so I'd size for peak throughput plus headroom rather than picking a huge number. I'd check key distribution for hot keys that would bottleneck one partition, and make sure we're on the cooperative-sticky rebalance protocol — default since 3.0 — so scaling consumers doesn't stop-the-world.

Weak: I'd set a high partition count, like 100, so we get lots of throughput and parallelism, and use a random or round-robin key so the load spreads evenly across all the partitions. That way every consumer stays busy and the stream is fast.