Skip to content

Chapter 2: Why HPC Networking Is Different · Part 2 of 6

Act 1 -- The mental model shift

In enterprise networking, a 1ms latency spike is a footnote in a monitoring dashboard. In an AI training cluster running a large language model across 256 GPUs, that same 1ms spike causes every GPU in the job to sit idle -- waiting for one straggler. The network is not just moving data. It is the synchronisation mechanism for a massively parallel computation.

Here is how the requirements compare side by side:

DimensionEnterprise networkingHPC / AI networking
Latency target1-10ms acceptableSub-2 microseconds required
Packet lossRare, recoverable via TCP retransmitZero tolerance -- stalls entire job
BandwidthShared, oversubscribedDedicated, non-blocking
Congestion handlingQoS queuing and selective dropPrevented with lossless fabric
Switch rolePure forwarding planeActive compute participant (SHARP)
NIC roleCommodity, kernel-attachedRDMA co-processor, kernel-bypassed

The right-hand column is not aspirational -- it is a hard requirement imposed by the way AI training works mathematically. The next two sections explain the math.