Chapter 13 filled in the context that fat-tree exists to solve. The torus topology -- which powered the fastest supercomputers in the world from 2004 to 2015 -- optimises for nearest-neighbour communication and eliminates switch cost. Its three failure modes for AI training are high hop count for distant nodes, low bisection bandwidth for all-to-all traffic, and limited fault-tolerant routing.
Dragonfly reduces cable complexity at petascale by using local all-to-all cliques connected by single global links, at the cost of global link bottlenecks under random traffic. Google's TPU Pod shows torus at its best -- a carefully folded 3D mesh with enormous per-chip bandwidth optimised for the spatial locality of convolutional workloads -- and illustrates where the workload mismatch with transformer all-to-all operations begins.
Fat-tree's bisection bandwidth guarantee -- non-blocking for any traffic matrix at 1:1 oversubscription -- is the property that wins for AllReduce-heavy training. Every architectural choice in the DGX fabric is downstream of that guarantee.