Inside a DGX H100 node, 4 third-generation NVSwitch chips provide 13.6 Tb/s of all-to-all bandwidth across 8 H100 GPUs. Every GPU can reach every other GPU in a single hop at 900 GB/s per GPU. This is the scale-up fabric -- NVLink, intra-node, designed for the tightest possible GPU-to-GPU communication.
Outside the DGX node, 8 ConnectX-7 HCAs connect to the external IB or RoCEv2 fabric at 400 Gbps each. A fully loaded DGX node contributes 3.2 Tbps of compute bandwidth to the external fabric. This is the scale-out fabric -- designed for communication between nodes.
NVLink — Intra-node vs Scale-out
Toggle between the two fabric tiers. The bandwidth cliff at the node boundary is the problem NVLink Switch solves.
All 8 GPUs are one NVSwitch hop away from each other. The 4 NVSwitch chips provide full all-to-all connectivity at 13.6 Tb/s aggregate. Every AllReduce that stays within this node runs at full NVLink bandwidth.
The two fabrics have completely different bandwidths, latencies, and addressing models:
| Scale-up (NVLink intra-node) | Scale-out (IB / RoCEv2) | |
|---|---|---|
| Bandwidth per GPU | 900 GB/s | 400 Gbps (50 GB/s) |
| Bandwidth ratio | 18x faster | baseline |
| Latency | ~1 us | 1-2 us |
| Addressing | NVLink address space | GID / GUID |
| Topology | Full-mesh via NVSwitch | Fat-tree |
| All-to-all (8 GPUs) | 13.6 Tb/s | -- |
For a single DGX node, this asymmetry does not matter -- all GPU-to-GPU traffic stays on NVLink. The problem emerges the moment a job needs more than 8 GPUs. Any AllReduce that crosses the node boundary immediately drops from 900 GB/s NVLink bandwidth to 50 GB/s IB bandwidth per GPU -- an 18x cliff.
This cliff has a name: the scale-up boundary. Every model parallelism decision, every tensor partition, every pipeline stage assignment is affected by where this boundary falls.