The Spectrum-4 ASIC (part number SN5600) integrates an Adaptive Routing engine directly into the forwarding pipeline. Unlike software-based traffic engineering, the AR engine operates at line rate - its decisions add zero latency to the forwarding path because queue-depth sampling runs in parallel with the route lookup, not in series with it.
How the AR engine makes forwarding decisions. Every candidate next-hop in an ECMP group exposes its current egress queue occupancy to the AR engine. Queue depth is sampled in hardware at sub-microsecond intervals. When a packet arrives that belongs to an ECMP group, the AR engine reads the current queue depth of each eligible next-hop and selects the one with the lowest occupancy. No hash is computed. The decision is purely reactive to real-time queue state.
This is fundamentally different from ECMP variants like WCMP (Weighted-Cost Multi-Path), which assigns static weights to links. AR weights are dynamic - they change every packet based on actual congestion state. A link that was lightly loaded 10 microseconds ago may be the most congested link now, and AR will already be avoiding it.
AR granularity modes. Spectrum-4 supports two AR granularities:
- Per-packet mode: every packet is independently routed to the best next-hop. Maximum load distribution, but produces out-of-order delivery. Requires BlueField-3 SuperNIC reorder buffer on hosts (DGX B200 only). This is the mode used in full SuperPOD deployments.
- Per-flowlet mode: packets within a flow are routed consistently until a configurable inactivity timeout (the flowlet timer) expires. After the timeout, the next packet in the flow is re-evaluated against current queue depths and may be redirected to a different next-hop. Flowlets preserve intra-flow ordering (packets within a burst stay in order) while still rebalancing across flows over time. ConnectX-7 clusters (DGX H100, H200) must use per-flowlet mode because CX7 lacks a hardware reorder buffer.
Queue depth feedback loop. The AR engine operates on a feedback loop: queue depth increases as the switch buffers packets, AR notices the increase and routes subsequent packets to less-loaded paths, queue depth decreases, other paths become eligible again. The loop converges in microseconds - faster than any software-driven traffic engineering system could respond. The 48MB shared buffer pool in Spectrum-4 provides enough headroom for the loop to operate without drops during convergence.
Protocol agnosticism. AR is lossless-agnostic: it works identically with RoCEv2, standard UDP, and TCP. The AR engine does not inspect above L3. This matters for mixed-workload fabrics where the same leaf switch carries both RoCEv2 training traffic and TCP storage traffic. Both benefit from AR without requiring separate configuration.
Interaction with PFC lossless queues. AR and PFC operate on orthogonal dimensions. PFC operates per-priority within a port (prevents buffer overflow on a specific queue class). AR operates across ports (routes packets to the least-loaded port). When AR redirects traffic to a different spine, the PFC priority of the packet is preserved end-to-end. A RoCEv2 packet in TC3 (PFC priority 3) routed by AR to a new spine arrives at the new spine still in TC3, still lossless.