There are four distinct approaches to load balancing, each with different trade-offs between visibility, accuracy, and implementation complexity.
Load balancing taxonomy — four modes, four trade-offs
Distributes individual packets across all available equal-cost paths. Each packet can take a different path independent of all previous packets from the same flow. Header entropy is irrelevant — path selection is based entirely on current port utilisation.
Per-packet — makes a fresh decision at line rate for every packet
Entropy-independent. Fixed dst port 4791, identical IPs — irrelevant. Every packet goes to the least-loaded port at that instant.
Solves both flowlet types fundamentally. Simultaneous flowlets: each packet is independently distributed. Sequential flowlets: no flow state to be stuck with — each packet is fresh.
Optimal. Elephant flows are automatically spread across all paths at the packet level. No flow-level concentration.
Load-balance mode: per-packet Adaptive: enabled Per-packet: enabled RSHP: enabled Reorder tolerance: 200 microseconds
IB per-packet adaptive routing (hardware — always active on QM9700+)
Yes — ConnectX-7 or later with reorder buffer
SLB -- Static Load Balancing
What it does: Assigns flows to paths using a static hash of the packet headers (the 5-tuple). Every packet in a flow follows the same path. The path is chosen once and never revisited.
Why it fails for AI: This is standard ECMP. All the problems described in Act 1 apply directly. Low entropy means most flows hash to the same paths. Elephant flows pile up on those paths while others are idle. No adaptation to real-time congestion.
When you still see it: Legacy Ethernet fabrics, any switch without DLB or GLB capability, and as the fallback when more sophisticated methods encounter issues.
DLB -- Dynamic Load Balancing
What it does: The switch monitors real-time bandwidth utilisation and buffer depth on each equal-cost output port. When a new flow arrives (or periodically for existing flows), the switch selects the output port with the lowest current utilisation rather than hashing to a fixed port.
The improvement over SLB: DLB can detect that a spine link is congested and route new flows to a less-loaded link. It responds to local conditions.
The limitation: DLB only has visibility into its own ports -- it knows how loaded its own uplinks are but cannot see what is happening at the next switch downstream. If all of Switch A's uplinks are healthy but Switch B's downlinks are congested, DLB on Switch A will route traffic toward Switch B without knowing it is sending into a bottleneck.
When to use it: Any AI fabric that cannot run per-packet spraying (see sDLB). DLB is a significant improvement over SLB and is available on most modern data center switches.
GLB -- Global Load Balancing
What it does: Extends DLB with next-to-next-hop (NNH) awareness. The switch knows not just the state of its own output ports but also the state of the links one hop further downstream. A leaf switch running GLB considers both the utilisation of its own uplinks to the spine AND the utilisation of the spine's uplinks to the far-side leaf switches.
The improvement over DLB: GLB avoids routing into a path that is locally uncongested but downstream-congested. It selects the path with the best end-to-end utilisation profile, not just the best local utilisation.
The limitation: GLB requires inter-switch communication to share utilisation state. This adds protocol overhead and latency to the path selection decision. At very high packet rates (400G), the state can go stale between updates.
When to use it: Large-scale fabrics with multiple spine tiers where spine utilisation is heterogeneous -- for example, a cluster where storage and compute traffic share some spine switches.
sDLB -- Selective Dynamic Load Balancing / Per-packet spraying
What it does: Instead of routing entire flows to a single path, sDLB sprays individual packets across all available equal-cost paths. Every packet can take a different path, independently of what the previous packet from the same flow did.
Why this solves the entropy problem fundamentally: Per-packet distribution does not depend on the 5-tuple at all. It does not matter that all RoCEv2 flows have the same destination port. Each packet is distributed to the least-loaded path at the moment it is forwarded. Low entropy in the headers becomes irrelevant.
The challenge this creates: Packets from the same flow can arrive at the destination out of order. If Packet 3 takes a slower path than Packet 4, Packet 4 arrives first. The receiver's RDMA engine, which expects packets in PSN order, sees a gap. Under native InfiniBand, this is handled by the protocol. Under RoCEv2, out-of-order arrival has historically caused the receiver to treat missing packets as dropped -- triggering retransmissions.
How NVIDIA Spectrum-X solves it: RSHP (Resilient Hashing and Per-packet reordering) adds a reorder buffer at the NIC. The ConnectX-7 in RoCEv2 mode can buffer out-of-order packets and present them to the RDMA engine in the correct sequence. This makes per-packet spraying viable on Ethernet without causing spurious retransmissions.
InfiniBand adaptive routing: InfiniBand has done per-packet routing since its inception. The Subnet Manager, which has complete topology visibility, programs the routing tables to measure per-port congestion and forward each packet to the least-loaded output port. This is what Ch0 Act 6 describes as the "adaptive routing vs ECMP" advantage.
When to use it: Any Spectrum-X deployment with RSHP-capable NICs (ConnectX-7 or later). Any InfiniBand deployment with adaptive routing enabled in UFM. This is the target state for production AI fabrics.