What Is SIMT, and Why Does Branch Divergence Slow a Warp Down?
SIMT, single instruction multiple threads, means the 32 threads of a warp share one instruction stream: each cycle the warp issues one instruction and every active thread executes it on its own data. When threads in a warp take different sides of a branch, the warp can't do both at once, so it runs one path with the other threads masked off, then the other path. The time is the sum of both paths, which is why divergence slows a warp down.
What you'll learn
- How SIMT differs from SIMD and from independent CPU threads
- What happens, lane by lane, when a warp diverges
- How much a divergent branch costs, worked through
- How to arrange a branch so that whole warps go the same way
Conventions and assumptions
CUDA terms and the warp size of 32 from the CUDA Programming Guide. The cost model counts instruction issue slots, assuming each path is a fixed number of instructions; real code also has memory effects.
SIMT vs SIMD
SIMD hardware, like a CPU's vector unit, makes the programmer pack data into vector registers and handle branching with masks explicitly. SIMT lets you write ordinary scalar code for one thread; the hardware groups threads into warps and applies the masks for you. The programming is simpler, but the cost model is the same underneath: one instruction, many lanes.
A divergent branch
__global__ void k(float *x) {
int lane = threadIdx.x % 32;
if (lane % 2 == 0) // even lanes
x[threadIdx.x] = pathA(x[threadIdx.x]);
else // odd lanes
x[threadIdx.x] = pathB(x[threadIdx.x]);
}Worked example: the cost of divergence
- Say pathA and pathB are 10 instructions each.
- In each warp, 16 lanes are even and 16 are odd, so both paths are needed.
- The warp issues pathA's 10 instructions with the 16 odd lanes masked off, then pathB's 10 with the even lanes masked off: 20 issue slots.
- Useful work: 10 × 16 + 10 × 16 = 320 lane-instructions out of 20 × 32 = 640 lane slots, so the lanes are 50% busy.
- Change the condition to (threadIdx.x / 32) % 2 == 0. Now whole warps go one way: each warp issues only 10 instructions with all 32 lanes busy, 100%.
When divergence doesn't matter
- The branch is uniform across the warp, for example it depends on blockIdx or on threadIdx.x / 32.
- One side is empty or tiny, such as the i < n bounds check: only the last warp diverges.
- The two paths are short compared with the memory accesses around them.
Common mistakes
- Branching on threadIdx.x % 2 or similar fine-grained patterns, which splits every warp.
- Branching on data values that vary randomly between neighboring threads; sorting or binning the data first can make warps uniform.
- Assuming a branch is free because only a few threads take it. If even one lane in a warp takes it, the whole warp spends the time.
- Relying on threads in a warp running in lockstep for correctness. Newer GPUs schedule threads more independently; use explicit synchronization.
Check your understanding
- Q1. A warp's lanes split 31 and 1 between two 10-instruction paths. How many issue slots? Answer: 20, the same as an even split.
- Q2. Does if (blockIdx.x < 10) cause divergence within a warp? Answer: no. Every thread in a block has the same blockIdx.
- Q3. Lanes 0 to 15 take a 20-instruction path, lanes 16 to 31 a 4-instruction path. Issue slots? Answer: 24.
Related
Last verified
Warp size and SIMT behavior checked against the CUDA Programming Guide. Every number in this post is computed in the site's tests (lib/gpuNumbers.test.js). Checked on 14 November 2026.