epoch.training← the field guide

Field Guide · Part II — Speed · 09

// gpu vs cpu

GPU vs CPU: the shape of the work

People say GPUs are "faster." That's the wrong frame. A GPU is not a fast CPU — it's a fundamentally different bet about what work looks like. Understand that bet and you understand why some code flies on a GPU and other code crawls, and why matrix multiply — the heart of every neural network — is exactly what a GPU was built for. It's about the shape of the work, not raw speed.

A CPU is designed to make a single stream of instructions finish as fast as possible. It spends most of its transistors not on arithmetic but on hiding latency for one thread: large caches, deep out-of-order pipelines, branch predictors, speculative execution. It has a handful of these very sophisticated cores. This is the right design when the work is a sequence of unpredictable, dependent decisions — parse this file, follow this pointer, branch on this condition.

A GPU makes the opposite bet. It assumes the work is thousands of near-identical operations with no dependence on each other — the same arithmetic applied to a huge array of data. So it spends its transistors on arithmetic units, not on latency-hiding cleverness, and packs in thousands of small, simple cores. It hides latency not by predicting the future but by having so many threads ready to run that when one stalls waiting on memory, another instantly takes its place.

SIMT: threads in lockstep

The mechanism NVIDIA calls SIMT — Single Instruction, Multiple Thread. Threads are grouped into warps of 32, and all 32 threads in a warp execute the same instruction at the same time, each on its own data element. One instruction fetch drives 32 lanes of arithmetic. That's how a GPU gets its throughput: it amortizes the expensive control logic across 32 threads instead of paying for it per thread like a CPU.

The catch is right there in the design. If those 32 threads hit an if and take different branches — branch divergence — the hardware can't run both paths at once. It runs the then-branch with the else-threads masked off doing nothing, then runs the else-branch with the others masked off. Divergent, branchy code turns a 32-wide machine into a 1-wide machine. This is precisely why the pointer-chasing, decision-heavy code that a CPU eats for breakfast runs terribly on a GPU — and vice versa.

Arithmetic intensity: the real limiter

Whether a GPU's arithmetic units are fed depends on arithmetic intensity — the ratio of math operations to bytes moved from memory. Moving a byte from memory costs far more time and energy than doing a floating-point operation, so a kernel that does little math per byte is memory-bound: the arithmetic units sit idle waiting for data, and the GPU's thousands of cores are wasted. To actually use a GPU you need high arithmetic intensity — lots of FLOPs per byte loaded.

Vector add:  c[i] = a[i] + b[i]
   loads 8 bytes, does 1 add  →  intensity ≈ 0.125 FLOP/byte
   → MEMORY-BOUND. Cores starve.

Matrix multiply (N×N):
   moves ~N² data, does ~N³ FLOPs  →  intensity ∝ N
   → each element loaded is reused N times
   → COMPUTE-BOUND. Cores stay fed. This is what GPUs want.
Matmul's arithmetic intensity grows with problem size: every value loaded is reused many times, so the ratio of math to memory traffic climbs — the opposite of a vector add.

Why matmul flies — and why AI runs on GPUs

Matrix multiplication is the ideal GPU workload on every axis. It is massively parallel — every output element is an independent dot product. It is branch-free — the same multiply-accumulate, no data-dependent ifs, so no warp divergence. And it has high, tunable arithmetic intensity — each input element gets reused across a whole row or column, so with the right tiling the arithmetic units stay saturated. A GPU built to run 32 identical operations in lockstep on data-parallel arrays is, almost by definition, a matrix-multiply engine.

And a neural network is, computationally, a stack of matrix multiplies. A transformer's attention and feed-forward layers are dense matmuls; training and inference are dominated by them. That is the whole reason the AI boom runs on GPUs and not CPUs: the fundamental operation of deep learning happens to be the exact shape of work GPUs were designed — originally for graphics — to devour. Modern accelerators lean into this so hard they add dedicated tensor cores that do nothing but small matrix multiplies. The hardware and the algorithm found each other.

Why it matters: "GPU vs CPU" isn't a speed contest, it's a question of matching work to machine. Latency-bound, branchy, pointer-chasing code belongs on a CPU; throughput-bound, regular, high-intensity array math belongs on a GPU. When someone says "just move it to the GPU and it'll be faster," the right question is: is this work actually GPU-shaped? For matrix multiply the answer is an emphatic yes — which is why the most valuable computation on earth right now is running on hardware originally built to draw triangles.

Sources & where to go deeper

NVIDIA, "CUDA C++ Programming Guide", NVIDIA Developer Docs — the SIMT Architecture section: warps of 32 threads, one instruction across all lanes, and how branch divergence serializes execution.
R. Thakur, R. Rabenseifner & W. Gropp, "Optimization of Collective Communication Operations in MPICH", Int. J. High Performance Computing Applications 19(1):49–66, 2005 — context for why the multi-GPU networking around these matmuls (page 06) is itself a bottleneck.

This is one page of twenty.

The workshops go deep on the real thing — scheduling, storage, interconnect, GPUs — hands-on, on real infrastructure, from someone who's run these machines at national scale.

See the trainings →