The enterprise assumption that fails

Standard load balancing -- ECMP -- was designed around a specific assumption: different flows will have different 5-tuples (source IP, destination IP, protocol, source port, destination port), and hashing those 5-tuples will distribute flows roughly evenly across available paths. For web traffic, this works well. Millions of clients connecting to thousands of services produce enormously varied 5-tuples. The hash distributes them evenly by construction.

AI training traffic violates this assumption structurally.

Consider what the 5-tuple looks like for two DGX nodes communicating during AllReduce on a 32-node cluster:

Source IP:        10.0.0.1  (one of 32 DGX node addresses)
Destination IP:   10.0.0.7  (one of 32 DGX node addresses)
Protocol:         UDP
Destination port: 4791      (fixed -- IANA-assigned RoCEv2 port, never changes)
Source port:      49183      (derived from Queue Pair number -- small range)

The destination port is always 4791. Every RoCEv2 packet from every GPU to every other GPU uses destination port 4791. This single fixed field eliminates most of the entropy that ECMP depends on. The source and destination IPs come from a pool of 32 addresses -- a far narrower range than web traffic. The source port derives from the QP number, which is a small integer.

The result: the ECMP hash function sees nearly identical inputs for all AllReduce traffic and produces nearly identical outputs -- routing all flows to the same spine links, leaving others idle. This is not bad luck. It is a structural property of RoCEv2 traffic.

Elephant flows: large, sustained, and indistinguishable

Individual AllReduce transfers are large. A gradient tensor for a 70B parameter model is measured in gigabytes. A single AllReduce step transfers that tensor from every GPU to every other GPU simultaneously. Each of these transfers is a sustained, high-bandwidth flow that lasts hundreds of milliseconds.

These large sustained flows are called elephant flows. In enterprise networks, elephant flows are the exception -- most traffic is bursty and short-lived. In AI training, every AllReduce produces elephant flows. The entire traffic matrix is elephant flows, running simultaneously, all with nearly identical headers.

When elephant flows hash to the same spine link, the resulting congestion is not brief. It lasts for the duration of the AllReduce -- hundreds of milliseconds. And because AllReduce runs continuously throughout training, the congestion is essentially permanent on those overloaded links.

Flowlets: when ECMP cannot tell flows apart

The low-entropy problem becomes even more acute at the flow level. A flowlet is an individual stream that a switch cannot distinguish from adjacent streams because their headers are identical.

Two types cause problems:

Simultaneous flowlets occur when multiple GPU pairs are communicating at the same moment. GPU 0 on Node A sends to GPU 0 on Node B. GPU 0 on Node C sends to GPU 0 on Node D. From the switch's perspective, examining only the IP/UDP headers, these may look identical -- same source port range, same destination port. The switch hashes both to the same path. Both flows are now competing for the same spine link.

Sequential flowlets occur when one job ends and another begins between the same GPU pair. Job 1 completes between GPU A and GPU B. Job 2 starts between the same two GPUs. The headers are byte-for-byte identical. The switch treats Job 2 as a continuation of Job 1's flow and keeps it pinned to the same path -- even if a better path became available during the pause between jobs. From the switch's perspective this looks like one long-lived elephant flow that never needs rebalancing.

East-west: why AI fabric is different from enterprise fabric

Enterprise data center traffic is predominantly north-south -- from servers toward the internet, from users toward applications. The traffic pattern is asymmetric: many clients (north) connecting to fewer servers (south), with most paths flowing toward the internet edge.

AI training traffic is almost exclusively east-west -- server to server, GPU to GPU, across the fabric interior. Every node is both a sender and a receiver simultaneously. The traffic matrix is dense and symmetric: every GPU is sending to every other GPU at the same time.

This changes where congestion occurs and how you read it:

In a north-south fabric, overloaded links are typically near the internet edge or core routers. Spine link utilisation is uneven by design -- some spines carry more north-south traffic than others based on routing decisions.

In an east-west AI fabric, every spine link should carry roughly equal traffic during AllReduce. Any asymmetry in spine link utilisation is a load balancing failure, not an expected design behaviour. When you open show interface counters on a spine switch and see links at 90% and 10% simultaneously during training, the fabric is not working correctly -- even if the 90% link is not dropping packets yet.