A consumer group can have at most one consumer per partition, so the partition count is the maximum useful number of consumers.
6 partitions, 6 consumers → each reads one
6 partitions, 10 consumers → 4 sit idle, forever
and the count can only go UP. increasing it changes
which partition a key hashes to, which breaks ordering
for existing keys across the boundary.
so: over-provision at creation. 12 partitions for a
topic needing 3 consumers today is cheap.The inability to reduce the count is what makes the initial decision matter, and the ordering break on increase is the subtler cost — a key that hashed to partition 2 may now hash to partition 7, so two messages for the same entity can be processed out of order during the transition. Over-provisioning is not free either: each partition is a set of files and adds to the broker’s memory and the rebalance time.