TCP was designed for reliability in the presence of packet loss -- its congestion window backs off when it detects a drop, retransmits the lost packet, and resumes. This is exactly what you need for a file transfer or a web request. It is exactly the wrong behaviour for AllReduce.
When TCP backs off, the GPU waits. When the GPU waits, the barrier stalls. The retransmit latency -- typically milliseconds -- is enormous relative to the sub-microsecond timing that AI fabrics require.
The replacement is RDMA (Remote Direct Memory Access) with RoCEv2 transport (RDMA over Converged Ethernet version 2). RDMA bypasses the CPU and kernel stack entirely: the NIC reads directly from GPU memory and writes directly into the remote GPU's memory with no CPU involvement and no kernel copies. The latency is measured in microseconds, not milliseconds.
But RDMA is brittle in one important way: it has no loss recovery mechanism. A dropped packet does not trigger a retransmit -- it triggers a queue pair error that requires application-level intervention to recover. This means RDMA requires the fabric beneath it to deliver every packet without loss. Which returns us to the same requirement: the fabric must be lossless.
The protocol stack for this chapter's context:
| Layer | Enterprise TCP stack | AI training stack |
|---|---|---|
| Application | Socket API | NCCL (AllReduce library) |
| Transport | TCP (loss-tolerant, slow recovery) | RDMA / RoCEv2 (lossless required) |
| Network | IP | IP + RoCEv2 header (UDP/4791) |
| Data link | Ethernet (lossy) | Ethernet + PFC (lossless) |
| Congestion | TCP backoff | ECN + DCQCN |
Figure: TCP and RDMA solve different problems. TCP spends time recovering from drops through the host stack; RDMA keeps latency low by using NIC-driven direct memory movement, which is why the Ethernet fabric underneath must be engineered to avoid loss in the first place.
The right column is what this curriculum builds toward. You now know why each row is different. The how comes in subsequent chapters.