The NVLink Switch System is not a replacement for IB or RoCEv2. It is an additional tier that makes sense for specific workload profiles and at specific scales. The decision depends on three factors: job size, model architecture, and memory requirements.
Scale-up vs Scale-out Decision
Select your job parameters to get a fabric recommendation with rationale.
Job size
Parallelism strategy
Workload type
Recommendation
NVLink Switch System (Scale-up)
Tensor parallelism requires all-to-all communication across all participating GPUs on every forward and backward pass. This is the workload NVLink Switch was designed for — 18× more bandwidth than IB means tensor parallel operations complete in microseconds instead of milliseconds.
⚠ At 128–256 GPUs this is the optimal configuration.
When NVLink Switch wins
Job fits within 256 GPUs: The NVLink Switch System's ceiling is 32 DGX H100 nodes / 256 GPUs. If your largest jobs stay within this bound, you get 18x more all-to-all bandwidth vs IB for the same scale.
Tensor-parallel-heavy workloads: Models that require tensor parallelism benefit most from the unified HBM pool and high all-to-all bandwidth. The elimination of tensor parallel communication can improve GPU utilisation by 10-30% on large transformer models.
Mixture-of-Experts (MoE) models: MoE architectures require expert-parallel communication patterns (all-to-all between all GPUs every forward pass). These patterns are disproportionately expensive on IB and disproportionately cheap on NVLink Switch.
When IB / scale-out wins
Jobs larger than 256 GPUs: NVLink Switch tops out at 256 GPUs. Training a 1T+ parameter model that requires 2,048 GPUs uses IB for inter-node communication regardless. NVLink Switch may still be present for intra-node (the 4 NVSwitch chips in each DGX node) but the inter-node fabric is IB.
Pipeline-parallel-heavy workloads: Pipeline parallelism sends activations stage-to-stage along a fixed pipeline. This is a unicast pattern, not all-to-all. IB handles it just as well as NVLink at much lower cost.
Mixed workloads across many tenants: IB's more mature isolation model (partition keys, LID assignments, UFM management) is better suited to a cluster serving many independent jobs with strict isolation requirements. NVLink Network isolation is hardware-enforced but the management tooling is less mature.
Cost: NVLink Switch modules are expensive. At the 32-node scale they become cost-competitive with the equivalent IB fabric when accounting for the elimination of IB switch ports, but below that scale they carry a cost premium.
The architecture decision in practice
Most large-scale AI training clusters today use a hybrid approach:
- NVLink (intra-node, always) -- the 4 NVSwitch chips inside every DGX H100 node
- NVLink Switch System (inter-node scale-up, for <=256 GPU jobs requiring high all-to-all BW)
- InfiniBand or RoCEv2 (inter-node scale-out, for >256 GPU jobs and storage)
The two external fabrics -- NVLink Switch and IB -- are not mutually exclusive. A DGX H100 node can have some ports committed to NVLink Switch and other ports committed to IB simultaneously, depending on the port configuration. The firmware determines which protocol each OSFP port speaks.