You have a problem that ECMP cannot solve cleanly.
In a standard fat-tree AI fabric running ECMP, all flows between a source and destination are distributed across available spine paths by hashing the 5-tuple (src IP, dst IP, src port, dst port, protocol). This works well for a mix of many small flows. It fails for GPU workloads in two specific and common ways.
The elephant flow collision problem
A 400GbE link carries roughly 47 GB/s of sustained throughput. An NCCL All-Reduce on 512 GPUs generates synchronised bursts where all GPU pairs transmit simultaneously. The 5-tuple hash is deterministic — given the same src/dst pair with the same port, the same spine is always selected. In an 8-spine fat-tree, the probability that two GPU pairs hash to the same spine is 1/8 per pair. At 512 GPUs with 256 flows, statistically 32 flows will share each spine. But the hash is not perfectly uniform in practice — entropy in NCCL port selection and flow sizes creates hot spines carrying 40–60 flows while others carry 10–20.
The consequence: one congested spine becomes the bottleneck for all GPU pairs hashing to it. The congestion control algorithm (Chapter 21) can reduce their send rate, but it cannot move the flow to a less-loaded spine. ECMP's hash is static per flow.
The elephant-and-mouse coexistence problem
AI fabrics carry two fundamentally different traffic types simultaneously:
- Training collectives (NCCL All-Reduce, All-to-All): latency-sensitive, bursty, sub-millisecond synchronisation requirements, typically 200–512 byte to 4 KB messages
- Checkpoint-to-storage (periodic model saves, dataset loading): throughput-hungry, latency-insensitive, message sizes 100 MB–10 GB, long-lived flows
When these two traffic types share the same spine pool under ECMP, the checkpoint flow becomes an elephant that occupies spine bandwidth needed by the collective. PFC pause propagation from the elephant's queue depth can spill into the collective's priority class. The collective's All-Reduce stalls. Training iteration time increases.
What Segment Routing provides
Segment Routing (SR) allows the source node or the headend switch to encode an explicit forwarding path directly in the packet header — without requiring any per-flow state in the core of the network. The path is carried as an ordered list of SIDs (Segment Identifiers) in the packet itself. Every switch in the path reads the next SID and forwards accordingly. No MPLS signalling, no RSVP-TE reservation, no stateful tunnels.
Traditional TE (RSVP-TE):
Controller → signals reservation → every switch in path stores flow state
When traffic stops: teardown signalling required
Scale: per-flow state in every switch = O(N×F) state
Segment Routing TE:
Headend encodes SID list in packet header
Core switches: stateless, read next SID and forward
Scale: per-policy state only at headend = O(P) state
For AI fabrics specifically, SR provides three capabilities that ECMP cannot:
- Explicit path steering per traffic class: checkpoint flows go over spines 4-8, collective flows go over spines 1-3. No interference.
- Flow rebalancing without rerouting: when a new training job starts, the controller can assign it to an underutilised path segment without touching running flows.
- Multi-tenant path isolation: Tenant A's training traffic follows SRv6 paths through VRF A's allocated segments; Tenant B is completely separated without VLAN proliferation.