The NVSwitch chip sits inside the DGX node (on the NVLink Switch Board, or NSB) and provides full all-to-all connectivity between all GPUs in the node. It is not the same as the external NVLink Switch System covered in Chapter 12 -- the onboard NVSwitch handles intra-node GPU-to-GPU traffic, while external NVLink Switch modules extend the NVLink domain across multiple nodes.

NVSwitch 1st generation (Volta / V100 DGX, 2018): 900 GB/s aggregate switch bandwidth. 6 DGX V100 nodes used this.

NVSwitch 2nd generation (Ampere / A100 DGX, 2020): 7.2 Tb/s aggregate switch bandwidth. 6 NVSwitch chips per DGX A100 node, each handling 12 NVLink 3 ports. 8 GPUs x 600 GB/s = 4.8 Tb/s total GPU bandwidth, 7.2 Tb/s switch headroom for full any-to-any.

NVSwitch 3rd generation (Hopper / H100 DGX, 2022): 13.6 Tb/s aggregate switch bandwidth across 4 NVSwitch chips in the DGX H100. Each GPU connects with 18 NVLink 4 links, providing 900 GB/s. This is the system we've been working with throughout the course. Also supports in-network SHARP-equivalent reduction.

NVSwitch 4th generation (Blackwell / B200, 2024): Higher aggregate bandwidth to match NVLink 5. The architectural change in Blackwell is the scale: external NVLink Switch modules now allow up to 72 GPUs (a full DGX B200 "NVL72" rack) to share a single NVLink address space, extending what was previously an 8-GPU intra-node fabric into a 72-GPU rack-scale fabric.

For the network engineer, the key implication of each NVSwitch generation: as intra-node bandwidth grows, the ratio of intra-node to inter-node bandwidth widens. More computation fits inside the NVLink domain without crossing the Ethernet or InfiniBand fabric. Your scale-out fabric carries a smaller fraction of total training traffic per generation -- but the jobs that do cross it are often larger models that need even more inter-node bandwidth.