epoch.training← the field guide

Field Guide · Part II — Speed · 14

// amdahl's law

Amdahl's law: where "just add nodes" stops

"It's slow? Throw more machines at it." That instinct is wrong more often than it's right, and there's a 1967 equation that tells you exactly when. The part of your program that can't be parallelized sets a hard ceiling on speedup — and no amount of hardware raises it.

Every program is a mix of work that can happen in parallel and work that must happen in sequence. Reading a config file, allocating the model, the final reduction that combines everyone's results — those are serial: one thing after another, no matter how many cores you own. Gene Amdahl, arguing in 1967 against the then-fashionable idea that you could reach any performance just by ganging up processors, wrote down the consequence. It's the single most important number in parallel computing, and most people never do the arithmetic.

The equation, and the number that stings

Let P be the fraction of your program that parallelizes and (1−P) the serial fraction that doesn't. With N processors, the parallel part runs in P/N of its original time, but the serial part is stuck at (1−P). So the speedup is:

              1
S(N) = ─────────────────
        (1 − P) + P / N

As N → ∞ :  S(∞) = 1 / (1 − P)

Serial fraction 5%  (P = 0.95):
   S(∞) = 1 / 0.05 = 20×      ← the ceiling

   N =   10 →  6.9×
   N =  100 → 16.8×
   N = 1000 → 19.6×   (you bought 1000, you got 20)
Amdahl's law. A mere 5% serial fraction caps you at 20× — forever. The last 900 processors buy you almost nothing.

Sit with that. Ninety-five percent of your code is perfectly parallel, and you still can't beat 20×. Push the serial fraction to 10% and the ceiling drops to 10×. This is why "scale it horizontally" quietly stops working: past a point, you're paying for processors that spend their lives waiting on the one part of the program that runs alone.

Gustafson's rebuttal: change the question

In 1988, John Gustafson — then at Sandia, working on a 1024-processor machine — pointed out that Amdahl assumes a fixed problem size. In practice, people who buy 1024 processors don't run yesterday's problem faster; they run a bigger problem in the same time. If the serial part stays roughly constant while the parallel work grows with the machine, the serial fraction shrinks, and useful speedup scales nearly linearly. This is weak scaling (fixed work per processor) versus Amdahl's strong scaling (fixed total work, more processors). Both are true. Which one governs you depends entirely on whether your problem grows with your hardware.

Why this rules distributed training

Data-parallel training is the textbook case. Each GPU processes a slice of the batch — beautifully parallel — but every step ends with an all-reduce that sums gradients across all GPUs before the next step can begin. That synchronization is Amdahl's serial fraction wearing a modern costume. Add GPUs and the per-GPU compute shrinks while the all-reduce cost grows with the group; eventually communication dominates and throughput plateaus. That plateau is why serious training runs lean on weak scaling (bigger global batches, bigger models as you add hardware — the Gustafson move) and pour engineering into the interconnect: overlapping communication with compute, tree/ring reductions, fatter fabric. When a lab says its cluster "doesn't scale past 4,000 GPUs," they've hit Amdahl on the all-reduce, not a bug.

Why it matters: before you approve a bigger cluster, ask what fraction of the workload is genuinely serial — the synchronization barriers, the checkpoint writes, the metadata step. That fraction, not the vendor's peak FLOPS, sets your real ceiling. Amdahl tells you when more nodes stop helping; Gustafson tells you how to change the problem so they help again. Knowing which regime you're in is worth more than the next hardware order.

Sources & where to go deeper

G. M. Amdahl, "Validity of the single processor approach to achieving large scale computing capabilities", AFIPS Spring Joint Computer Conference 1967, pp. 483–485 (DOI 10.1145/1465482.1465560) — the original argument; the law is named for it.
J. L. Gustafson, "Reevaluating Amdahl's Law", Communications of the ACM 31(5), May 1988, pp. 532–533 (DOI 10.1145/42411.42415) — the weak-scaling rebuttal, now Gustafson's law.

This is one page of twenty.

The workshops go deep on the real thing — scheduling, storage, interconnect, GPUs — hands-on, on real infrastructure, from someone who's run these machines at national scale.

See the trainings →