Skip to content

Chapter 0: How We Got Here -- The Hardware Story · Part 7 of 15

Act 5 -- What the industry tried, and what won

Building the model…

Follow one RDMA write from GPU memory to GPU memory across two scalable units. The header strip lights what each device reads at that hop; step through it or let it play.

The journey to today's AI networking infrastructure went through several stages.

Industry evolution — click any era

InfiniBand adoption begins

Supercomputer tech enters AI clusters

2016–2018

Models outgrew single GPUs. Researchers discovered InfiniBand — which had existed in supercomputing since the early 2000s — and adopted it for GPU clusters. InfiniBand's credit-based flow control made it naturally lossless. GPUDirect RDMA allowed GPU memory to be directly exposed to the InfiniBand fabric. The V100 (2017) had 300 GB/s NVLink and full InfiniBand support.

InfiniBand EDR (100G)GPUDirect RDMANVLink gen2Mellanox ConnectX-4/5
Why this happened: InfiniBand was battle-tested in supercomputing for exactly this use case — lossless, low-latency interconnect for tightly coupled parallel workloads.

2010-2015: Early deep learning clusters Researchers used standard Ethernet with TCP/IP. Training was slow but the models were small. The network was not yet the bottleneck.

2016-2018: The GPU cluster era begins Models got larger. Multi-GPU training became necessary. Teams started using InfiniBand -- a technology that already existed in supercomputing -- for GPU-to-GPU communication. InfiniBand was designed from the start to be lossless, with credit-based flow control built into the protocol.

2019-2020: NVIDIA acquires Mellanox In March 2020, NVIDIA completed its $6.9 billion acquisition of Mellanox Technologies -- the dominant maker of InfiniBand switches and NICs. This was a strategic move: NVIDIA controlled the GPU, and now it controlled the network that connected GPUs. The DGX reference architecture became an integrated product.

2021-2023: RoCEv2 emerges as an alternative RoCEv2 (RDMA over Converged Ethernet v2) lets you run RDMA -- the same kernel-bypass technology as InfiniBand -- over standard Ethernet. For enterprises that already had Ethernet infrastructure, this was attractive. But it required making Ethernet lossless using PFC and ECN -- which is exactly what this course teaches.

2024-present: Two camps Today the industry has effectively split:

  • InfiniBand (Quantum-2 NDR, Quantum-X800 XDR) -- used for hyperscale AI training clusters, cloud providers building dedicated AI infrastructure, and any deployment where absolute maximum performance matters
  • Spectrum-X (RoCEv2 over Ethernet) -- used where Ethernet infrastructure already exists, for enterprises wanting multi-tenant GPU clusters, or where the operational simplicity of Ethernet outweighs the raw performance of InfiniBand

Neither is "wrong." The choice depends on workload, budget, and existing infrastructure.


Read aloudPage 7