Adaptive routing is not unique to Spectrum-X. InfiniBand has provided AR in ASIC hardware for multiple generations. Understanding the architectural differences is essential for both the NCP-AIN exam and for guiding customers in platform selection.
InfiniBand AR: credit-based architecture. IB fabrics use a credit-based flow control system (described in Ch8). Credits are pre-allocated per Virtual Lane (VL) per port. A sender cannot transmit unless it holds credits from the receiver. This architecture has a profound implication for AR: because the credit system prevents buffer overflow at the receiver by design, out-of-order delivery between virtual lanes does not cause NAKs. IB AR can spray packets across multiple VLs freely, without a reorder buffer on the host.
The IB Subnet Manager (OpenSM or NVIDIA UFM, covered in Ch13) orchestrates AR using the dfsssp routing algorithm, which computes minimum-congestion paths considering the full fabric topology. This is a centralized control-plane approach: the SM knows the entire fabric, computes optimal paths, and programs them into switch Local Routing Headers (LRH). When a link becomes congested, the SM recomputes paths and pushes updates.
Spectrum-X AR: distributed hardware approach. Spectrum-X AR is entirely distributed. Each Spectrum-4 switch makes per-packet forwarding decisions independently based on local queue depth measurements. There is no central controller involved. This makes Spectrum-X AR faster to react (microsecond vs millisecond for SM path recomputation) but requires the host-side reorder buffer (BF3 SuperNIC) to handle OOO.
Key comparison table:
Property | Spectrum-X AR | IB AR
--------------------------|------------------------|----------------------
Control plane | Distributed (per-ASIC) | Centralized (SM)
Reaction time | Microseconds | Milliseconds
OOO handling | BF3 SuperNIC buffer | Built into credit system
Host NIC requirement | BF3 (B200) or | Standard IB HCA
| flowlet mode (CX7) |
Protocol support | RoCEv2, UDP, TCP | InfiniBand native
ECMP group size | 512/1024 buckets | VL-based routing
Failure recovery | Resilient hashing | SM reroutes
Configuration interface | NVUE | SM policy + UFM
When to choose each. For pure GPU training workloads requiring maximum collective bandwidth with DGX B200 hosts, per-packet Spectrum-X AR with BF3 reorder is competitive with IB AR. For mixed DGX H100/H200+B200 clusters, or clusters requiring strict per-flow ordering guarantees, IB AR or per-flowlet Spectrum-X AR is more appropriate. For multi-tenant GPU clouds where isolation is paramount alongside performance, Spectrum-X AR combined with BGP-EVPN (Ch27) is the preferred architecture.
This chapter completes the core Spectrum-X operational stack: Ch24 established hardware, Ch25 configured RoCE, and Ch26 added adaptive routing for optimal load distribution. Ch27 will layer BGP-EVPN multi-tenancy on top of this foundation.