Skip to content

Chapter 0: How We Got Here -- The Hardware Story · Part 7 of 15

Act 5 -- What the industry tried, and what won

The journey to today's AI networking infrastructure went through several stages.

Industry evolution — click any era

InfiniBand adoption begins

Supercomputer tech enters AI clusters

2016–2018

Models outgrew single GPUs. Researchers discovered InfiniBand — which had existed in supercomputing since the early 2000s — and adopted it for GPU clusters. InfiniBand's credit-based flow control made it naturally lossless. GPUDirect RDMA allowed GPU memory to be directly exposed to the InfiniBand fabric. The V100 (2017) had 300 GB/s NVLink and full InfiniBand support.

InfiniBand EDR (100G)GPUDirect RDMANVLink gen2Mellanox ConnectX-4/5
Why this happened: InfiniBand was battle-tested in supercomputing for exactly this use case — lossless, low-latency interconnect for tightly coupled parallel workloads.

2010-2015: Early deep learning clusters Researchers used standard Ethernet with TCP/IP. Training was slow but the models were small. The network was not yet the bottleneck.

2016-2018: The GPU cluster era begins Models got larger. Multi-GPU training became necessary. Teams started using InfiniBand -- a technology that already existed in supercomputing -- for GPU-to-GPU communication. InfiniBand was designed from the start to be lossless, with credit-based flow control built into the protocol.

2019-2020: NVIDIA acquires Mellanox In March 2020, NVIDIA completed its $6.9 billion acquisition of Mellanox Technologies -- the dominant maker of InfiniBand switches and NICs. This was a strategic move: NVIDIA controlled the GPU, and now it controlled the network that connected GPUs. The DGX reference architecture became an integrated product.

2021-2023: RoCEv2 emerges as an alternative RoCEv2 (RDMA over Converged Ethernet v2) lets you run RDMA -- the same kernel-bypass technology as InfiniBand -- over standard Ethernet. For enterprises that already had Ethernet infrastructure, this was attractive. But it required making Ethernet lossless using PFC and ECN -- which is exactly what this course teaches.

2024-present: Two camps Today the industry has effectively split:

  • InfiniBand (Quantum-2 NDR, Quantum-X800 XDR) -- used for hyperscale AI training clusters, cloud providers building dedicated AI infrastructure, and any deployment where absolute maximum performance matters
  • Spectrum-X (RoCEv2 over Ethernet) -- used where Ethernet infrastructure already exists, for enterprises wanting multi-tenant GPU clusters, or where the operational simplicity of Ethernet outweighs the raw performance of InfiniBand

Neither is "wrong." The choice depends on workload, budget, and existing infrastructure.