ProDeploy — Learn the fabric that keeps AI clusters alive

Connecting 256 GPUs into a flat NVLink fabric creates an addressing problem. NVLink uses physical addresses in GPU HBM, not routable IP or IB GIDs. At 256-GPU scale with multiple jobs potentially sharing the system, hardware address translation is required.

NVLink Network address space

The NVLink Switch System introduces a NVLink Network address space -- a virtualised address layer that sits above the physical HBM addresses. Each GPU is assigned a NVLink Network address. The NVSwitch hardware performs address translation at line rate, mapping NVLink Network addresses to physical HBM addresses in the target GPU.

This translation enables tenant isolation: two jobs running on the same NVLink Switch System fabric cannot reach each other's memory even though they share the same physical switches. The NVSwitch hardware enforces the address space boundary -- there is no software routing step that could be misconfigured.

This is a meaningful difference from IB partition keys (P_Keys), which isolate traffic at the Subnet Manager level and can be misconfigured. NVLink Network isolation is enforced in hardware at the switch ASIC.

SHARP on NVSwitch at 256-GPU scale

Chapter 0 described SHARP (Scalable Hierarchical Aggregation and Reduction Protocol) in the context of InfiniBand -- the IB switch performing AllReduce reduction in-network as gradients flow through it, reducing the traffic that reaches GPUs by half.

The NVSwitch SHARP equivalent operates at a different level. At 256-GPU scale, in-network reduction across 256 H100 SXM5 GPUs yields approximately 192 TFLOPS of in-network compute. This is not the IB SHARP covered in Chapter 0 -- it operates on NVLink traffic within the switch fabric, not on IB traffic. The two SHARPs are complementary in a system that has both fabrics, but they operate on different traffic types and at different scales.

The 192 TFLOPS figure is significant: it represents in-network compute capacity that exceeds many entire GPU clusters built only a few years ago. For AllReduce-heavy workloads like large language model training, offloading the gradient summation to the switch fabric reduces GPU stall time and improves FLOP/s utilisation.