Skip this section unless you are validating GB300 NVL72 racks or planning for the Rubin generation.

A rack-scale system adds a scale-up domain that must be validated separately. In a GB300 NVL72, seventy-two GPUs are joined by nine NVLink switch trays into one all-to-all domain at 130 TB/s aggregate. Collectives that fit inside 72 GPUs never touch the scale-out fabric at all. This is excellent for performance and confusing for acceptance, because an all-reduce at 72 GPUs is measuring NVLink, not your Ethernet or InfiniBand fabric, and it will produce a number so much higher than a node-based cluster that it is easy to mistake for a passing scale-out result.

Validate the two domains as two things:

# Scale-up: inside the rack, before any scale-out traffic
$ nvidia-smi nvlink --status                 # every link at rate
$ nvidia-smi nvlink -e                       # error counters, expect zero
$ nvbandwidth -t all_to_all_bidirectional    # NVLink domain bandwidth
$ srun -N 1 --gpus=72 ... all_reduce_perf -b 8 -e 16G -f 2   # NVLink-only

# Scale-out: force traffic across racks, which is the fabric under test
$ srun -N <2 racks> --ntasks-per-node=... all_reduce_perf -b 8 -e 16G -f 2
$ srun -N <all racks> ... alltoall_perf -b 1M -e 8G -f 2

The scalable unit is redefined. When one rack is 72 GPUs on a scale-up fabric, the leaf-local group that used to be your unit is now a group of racks, and the criteria that referenced "no spine crossing inside a unit" have to be restated in terms of the new boundary. Rewrite the node lists and the criteria table before the campaign starts, not during it. The consistency criterion becomes rack-to-rack rather than unit-to-unit, and it is if anything more valuable, because a rack is a large amount of capacity to have quietly running 5 % slow.

Scale-out per GPU is unchanged in kind. Each GPU still presents 800 Gb/s to the scale-out fabric through a ConnectX-8 SuperNIC, so link qualification, rail auditing and the whole Phase-1 methodology transfer directly. What changes is the ratio: far more of the collective traffic is absorbed inside the rack, so the scale-out fabric sees a different mix — fewer, larger, more bursty flows crossing the spine. Expect the all-to-all result to be more sensitive to adaptive routing than it was on a node-based cluster, and weight it accordingly.

Rubin-generation notes. ConnectX-9 doubles scale-out to 1.6 Tb/s per GPU, and the line-rate figure in your link-qualification thresholds changes with it. Spectrum-6 switches with co-packaged optics remove the pluggable transceiver from the fault domain, which changes what a failing link qualification means: there is no module to reseat, so a bad result points at fibre, connector cleanliness, or the switch itself. NVLink 6 raises the scale-up domain's bandwidth again, which pushes the balance of traffic further inside the rack. The methodology does not change. The thresholds, the node lists and the reference tables all do — and they must be re-derived from the current reference architecture rather than carried forward.