RoCEv2 per-packet spraying (RSHP in Spectrum-X, Ch6 Act 6) requires NIC firmware support to vary the source UDP port on each packet, so ECMP hashes differently per packet. The switch sees different 5-tuples and distributes across paths. But this is an approximation: the hash function distributes with statistical fairness, not precise per-packet round-robin.

UEC's multipath mechanism is specified at the transport layer. The sender explicitly assigns packets to paths. The protocol defines a set of path IDs that correspond to available ECMP next-hops. The sender NIC cycles through available path IDs, assigning successive packets to successive paths. The receiver uses Message ID and Packet Offset to reassemble regardless of arrival order.

The path assignment is in the packet header as a hint to ECMP-aware switches. A UEC-capable switch can use the path hint field for forwarding decisions rather than (or in addition to) the 5-tuple hash. This gives deterministic path assignment rather than hash-probabilistic assignment.

Explicit path awareness

UEC switches export path utilisation metrics to endpoints via a dedicated feedback channel (defined in the UEC spec as the Network Performance Monitor, NPM). The NIC can receive per-path utilisation data from the fabric and dynamically select less-loaded paths. This closes the loop: the NIC knows which paths are congested before sending, not after.

In a 32-node BasePOD with 8 spine switches, each DGX node has 8 ECMP paths to every other node. Under UEC with NPM, the sending NIC receives per-spine utilisation from each spine switch and assigns gradient AllReduce packets to the least-loaded spine at the time of sending. This is more effective than GLB (which learns about congestion from CNPs, after the fact) because the information arrives before the packet is sent.

Receiver buffer requirements

Out-of-order arrival under multipath spraying requires receiver buffer to hold the maximum in-flight data for any given message. For a 256KB gradient block sprayed across 8 paths, the maximum out-of-order window is the bandwidth-delay product across one path times the number of paths minus one. At 400G x 8 paths x 2us RTT, the maximum buffering per message is approximately 800 KB. UEC NICs must provision receive buffers accordingly -- this is a silicon cost that does not exist in ordered RoCEv2.