Standard Ethernet was not designed for GPU collectives. When you run NCCL All-Reduce across 1,024 GPUs over conventional Ethernet, three failure modes converge simultaneously. ECMP hash collisions place multiple elephant flows onto the same spine link while adjacent links sit idle — the hash function has no visibility into queue depth, so it cannot self-correct. Head-of-line blocking in switch port buffers causes a single large flow to stall smaller flows sharing the same egress queue. Most critically, PFC storm risk: when a receiver asserts PAUSE on a priority class, the pause propagates upstream hop by hop, eventually freezing unrelated flows in a deadlock cascade. These pathologies combine to produce effective utilisation well below 50% of installed bandwidth during All-Reduce synchronisation barriers.
Spectrum-X is NVIDIA's answer: a vertically integrated, AI-optimised Ethernet platform that addresses all three pathologies simultaneously through hardware co-design rather than software workarounds. The platform has four components that must be present together to unlock its full capability. The Spectrum-4 switch ASIC provides the hardware adaptive routing engine that eliminates ECMP hash collisions at the packet level. The BlueField-3 SuperNIC (in DGX B200 only — never in H100 or H200) provides the in-NIC reorder buffer that makes per-packet adaptive routing viable without triggering RoCEv2 Go-Back-N retransmit storms. The DOCA SDK exposes programmable packet processing on the BF3 ARM cores. NetQ provides fabric-wide telemetry with nanosecond-resolution event capture.
The "AI factory" framing matters for how you think about cabling and topology. A production AI factory has three distinct fabrics with different traffic characteristics. The training fabric carries GPU-to-GPU collective traffic — bandwidth-sensitive, bursty, synchronous, and absolutely intolerant of head-of-line blocking. This is where Spectrum-4 leaf and spine switches live. The storage fabric carries checkpoint and dataset traffic — throughput-sensitive but more tolerant of latency jitter, typically terminated on dedicated SN4600C storage leaf switches. The OOB management fabric carries BMC, console, and orchestration traffic — lowest bandwidth, completely isolated from training and storage. Spectrum-X components span all three fabrics but in different roles.
A standard Spectrum deployment — same Spectrum-4 ASIC, same Cumulus Linux — without the BF3 SuperNIC is just a high-bandwidth switch. The Spectrum-X designation means the full stack is present and the adaptive routing engine is configured to use per-packet spraying with the BF3 reorder buffer absorbing out-of-order delivery at the endpoint.
# Verify you are looking at a Spectrum-X deployment (not standard Spectrum)
leaf-01# decode-syseeprom | grep -E "Product|Part"
Product Name: MSN5600-CS2FO
Part Number: 920-9N110-00R7-0N0
# SN5600 = Spectrum-4 ASIC confirmed
# Check that adaptive routing is in spectrum-x mode (not standard ECMP)
leaf-01# nv show router adaptive-routing
operational applied
------------------ ----------- -------
enable on on
mode spectrum-x spectrum-x
reorder-mode endpoint endpoint