Meaning
Uneven data distribution across the partitions of a distributed database or stream processing system creates severe processing bottlenecks. When a single partition receives a disproportionately large amount of data compared to the others, partition skew occurs. This imbalance results in a single node performing most of the work while other nodes remain idle.
Distributed systems must manage the partition keys to avoid this imbalance.
Data Distribution
Choosing a partition key with low cardinality represents the primary cause of this data imbalance. The presence of partition skew can be observed when a few worker nodes consume more CPU and memory than their peers. This situation can cause some nodes to run out of memory.
The overall throughput of the system is limited by the slowest worker node.
Performance Impact
Performance degradation is inevitable when the processing of a parallel job is blocked by a single task. This delay caused by partition skew prevents the job from completing in a timely manner. The system cannot scale effectively because adding more nodes does not resolve the imbalance.
Developers must identify the skewed keys to resolve the issue.
Mitigation Strategy
Salting the partition keys by adding a random suffix can distribute the data more evenly across the cluster. This technique mitigates partition skew by breaking up large data blocks before they reach the workers. The final assembly achieves balanced processing across all nodes.