Why Adding Processors Stops Helping

Every job that gets split across multiple processors has two kinds of parts. Some steps run independently and can be handed to separate processors at once. Other steps must happen in order, one after another, because a later step needs the result of an earlier one. That ordered stretch is the sequential portion, and no number of extra processors can shrink it. You can split the parallel part across 2, 20, or 2000 processors and watch its time drop, but the sequential part just sits there, waiting its turn.

That fixed sequential chunk is why speedup bends into a flat curve instead of climbing in a straight line. Early on, adding processors slashes the large parallel portion and total time drops fast. But as the parallel part keeps shrinking toward zero, the sequential part becomes a bigger and bigger share of what's left. Eventually almost all the remaining time is that unsplittable chunk, and piling on more processors barely moves the total. The theoretical best speedup, once you have unlimited processors, is 100 divided by the percent that's sequential.

This idea connects directly to why real systems favor many moderately fast processors over chasing one impossibly fast one, and why programmers spend real effort hunting for ways to shrink or restructure the sequential portion itself rather than just buying more hardware. It also explains why distributed systems and networks, the other topics in this unit, are designed around tolerating bottlenecks rather than pretending they can be erased.