The NVSwitch chip sits inside the DGX node (on the NVLink Switch Board, or NSB) and provides full all-to-all connectivity between all GPUs in the node. It is not the same as the external NVLink Switch System covered in Chapter 12 -- the onboard NVSwitch handles intra-node GPU-to-GPU traffic, while external NVLink Switch modules extend the NVLink domain across multiple nodes.

NVSwitch 1st generation (Volta / V100 DGX, 2018): 900 GB/s aggregate switch bandwidth. 6 DGX V100 nodes used this.

NVSwitch 2nd generation (Ampere / A100 DGX, 2020): 7.2 Tb/s aggregate switch bandwidth. 6 NVSwitch chips per DGX A100 node, each handling 12 NVLink 3 ports. 8 GPUs x 600 GB/s = 4.8 Tb/s total GPU bandwidth, 7.2 Tb/s switch headroom for full any-to-any.

NVSwitch 3rd generation (Hopper / H100 DGX, 2022): 13.6 Tb/s aggregate switch bandwidth across 4 NVSwitch chips in the DGX H100. Each GPU connects with 18 NVLink 4 links, providing 900 GB/s. This is the system we've been working with throughout the course. Also supports in-network SHARP-equivalent reduction.

NVSwitch 4th generation (Blackwell and Blackwell Ultra / B200, B300, 2024-2025): higher aggregate bandwidth to match NVLink 5, at 14.4 TB/s across the eight-GPU board. The architectural change in Blackwell is scale: in the rack-scale Grace-Blackwell form, NVLink switch trays join 72 GPUs into one address space, turning what used to be an 8-GPU intra-node fabric into a 72-GPU rack-scale one. On an HGX B200 or HGX B300 board the domain is still the eight GPUs on that board.

For the network engineer, the key implication of each NVSwitch generation: as intra-node bandwidth grows, the ratio of intra-node to inter-node bandwidth widens. More computation fits inside the NVLink domain without crossing the Ethernet or InfiniBand fabric. Your scale-out fabric carries a smaller fraction of total training traffic per generation -- but the jobs that do cross it are often larger models that need even more inter-node bandwidth.