Rail-optimised design (ROD) -- what we have been teaching
In all previous chapters, every DGX node has 8 NICs connecting to 8 different leaf switches -- one NIC per rail. Leaf Switch 0 connects only to GPU 0 across all nodes. Leaf Switch 1 connects only to GPU 1. And so on.
This means AllReduce between GPU 0s on all nodes stays on Leaf Switch 0 -- a single switch hop. GPU 0 on Node 1 sends to Leaf Switch 0; Leaf Switch 0 forwards to GPU 0 on all other nodes. The spine switches are never involved for intra-GPU-index AllReduce.
The diagnostic benefit of ROD: When you see a problem on Rail 3, you know exactly which GPU index is affected on every node (GPU 3) and exactly which leaf switch is the focus (Leaf Switch 3). The mapping is unambiguous. This is why Ch3 Lab 0's topology shows Rail 3 as a distinct failing entity -- it means one GPU on every node is affected, not all GPUs on one node.
Rail-optimised (ROD) vs rail-unified (RUD) wiring
1 switch hop — GPU 0 on Node A to GPU 0 on Node B goes through Rail 0 switch only. Spine not involved.
A failing rail affects GPU index N across all nodes. Diagnostic is unambiguous: 'Rail 3 is down' = GPU 3 on every node.
GPU 0 cannot AllReduce with GPU 1 without going through the spine. Cross-rail traffic uses more hops.
Fixed: each GPU gets exactly 400G to its dedicated rail. No sharing possible between GPU indices.
Rail-unified design (RUD)
In a rail-unified deployment, all 8 NICs of a DGX node connect to the same set of leaf switches -- typically 2 or 4 switches, each carrying some fraction of the node's bandwidth. All 8 GPUs on a node appear on the same leaf switches.
The bandwidth aggregation benefit: For workloads that generate large traffic volumes from one node to another (rather than GPU-index-to-GPU-index AllReduce), RUD provides more bandwidth between a pair of nodes. Two nodes that are each connected to the same 4 leaf switches can exchange data across 4 x 400G = 1.6 Tb/s of parallel paths -- more than the 400G a single rail can carry.
When enterprises choose RUD:
- Mixed workloads: training and inference running simultaneously, where inference traffic does not follow GPU-index patterns
- Large model parallelism: pipeline parallelism that sends large tensors between whole nodes rather than between specific GPU indices
- Multi-tenant clusters: multiple jobs running simultaneously, where no single job owns a full rail
The diagnostic cost of RUD: When a port fails in RUD, the impact is not a clean "Rail 3 is down." Some fraction of every GPU's bandwidth is affected. Counter patterns are harder to read -- which switch, which port, which GPU. RUD trades diagnostic clarity for bandwidth flexibility.
The current course scope: All labs and chapters use ROD, which is the dominant design for dedicated AI training clusters. RUD becomes relevant as clusters move toward mixed training/inference and multi-tenant workloads.