Skip to content

Chapter 6: Efficient Load Balancing -- Preventing Congestion Before It Starts

HPC Networking FoundationsIntermediate50 min read

Chapters 3 and 5 taught you to read congestion -- the counters that reveal it, the mechanisms that contain it. PFC pauses senders before buffers overflow. ECN signals senders to reduce rate before PFC is needed. You understand both mechanisms well.

But there is a layer that sits above both, and it is the one that should make PFC and ECN unnecessary most of the time. Load balancing. If the fabric distributes AllReduce traffic correctly across all available paths, no single link ever fills. ECN thresholds are never crossed. PFC frames are never sent. Training proceeds at line rate without interruption.

When you see PFC counters growing continuously in production -- not occasional bursts, but sustained, constant pausing -- that is usually not a PFC configuration problem. PFC is working exactly as designed. The problem is that load balancing is failing to distribute traffic, so one path is permanently congested while adjacent paths sit idle. Tuning PFC thresholds will not fix this. Understanding why the traffic ended up concentrated on one path, and how to fix the distribution, is what this chapter is about.

The picture from Chapter 5 is now complete:

Congestion management stack

Load balancing is proactive. It acts on every packet decision, before any buffer fills. ECN is the early warning. PFC is the emergency brake. You want the emergency brake to be almost never needed.