Skip to content

Chapter 14: GPU Hardware Generations and Network Implications · Part 11 of 13

Act 10 -- Blackwell Ultra (B300): the reference platform for this course

Building the model…

Heatsinks lifted: each B300 is two dies surrounded by eight HBM3e stacks, 288 GB per GPU. Switch the traffic mode to see where each network leaves the board.

Everything from Chapter 0 onward is illustrated against a working cluster, and the platform that cluster is built on is HGX B300. It is worth stating its numbers once, in one place, because most of the fabric decisions in the later chapters follow directly from them.

Per GPU. Blackwell Ultra carries 288 GB of HBM3e. Dense tensor-core throughput is roughly 2.25 PFLOPS at BF16, 4.5 PFLOPS at FP8 and 13.5 PFLOPS at NVFP4 -- the same three figures the DGX Cloud benchmarking recipes use when they compute model FLOPs utilisation, so they are the numbers you will find yourself dividing by during acceptance. NVLink 5 gives each GPU 1.8 TB/s to its siblings.

Per board. An HGX B300 carries eight Blackwell Ultra SXM modules on one baseboard, about 2.1 TB of HBM3e in total, with 14.4 TB/s of aggregate NVLink bandwidth across the onboard switch. Dense board-level throughput is therefore around 18 PFLOPS BF16, 36 PFLOPS FP8 and 108 PFLOPS NVFP4.

The network implication, which is the part that matters here. Scale-out attachment doubles relative to B200: eight ConnectX-8 SuperNIC ports at 800 Gb/s each, one per GPU, rail-optimised. NVIDIA quotes that as 1.6 TB/s of networking bandwidth per node against 0.8 TB/s for HGX B200 -- bidirectional figures, so the one-way arithmetic is 8 x 800 Gb/s = 6.4 Tb/s out the back of every node.

Three consequences cascade from that single change:

Your leaf tier doubles in cost per node. Eight 800 Gb/s ports per node instead of eight 400 Gb/s ports means either twice the switch radix at the same port count, or twice the switches. The rail-optimised sizing arithmetic in Chapter 31 works the same way; every number in it is simply larger.

The optics change generation. 800 Gb/s per rail moves the physical layer to OSFP at 8 x 100G-PAM4 lanes, with all the reach, power and thermal consequences Chapter 9 covers. Twin-port modules that carry two 800 Gb/s channels in one cage become the normal way to build the leaf-to-spine tier, because a full-radix leaf otherwise runs out of physical cages before it runs out of ports.

Congestion behaviour gets less forgiving, not more. Doubling per-endpoint bandwidth without doubling buffer depth means the same incast event fills a queue in half the time. The PFC and ECN thresholds from Chapter 5, and the adaptive-routing behaviour from Chapter 26, matter more at 800 Gb/s than they did at 400.

GPU form factors: SXM vs PCIe
Toggle to compare connectivity paths and bandwidth in each form factor
GPU-to-GPU bandwidth
SXM: 900 GB/s (NVLink 4)
PCIe: 31.5 GB/s (PCIe Gen 4)
SXM advantage: 28×
NIC-to-GPU (RDMA)
SXM: Full line rate (GDS)
PCIe: PCIe ceiling limited
SXM advantage: ~6×
NVLink Switch eligible
SXM: Yes — full port count
PCIe: No — NVLink absent
Primary workload
SXM: Training at scale
PCIe: Inference / cloud VM
SXM connectivity paths
GPU A (HBM)→ via NVSwitch →GPU B (HBM)
900 GB/s
GPU→ via NVLink → CPU PCIe →ConnectX-7
400 Gbps NIC
DPU→ via NVMe-oF / GDS →NVMe storage
800 Gbps
CPU→ via PCIe Gen 5 (control only) →GPU
63 GB/s
SXM: NVLink handles all GPU-to-GPU traffic. PCIe carries only CPU control traffic and ConnectX-7 storage NIC (Slot1/Slot2) operations — neither on the training critical path.
Read aloudPage 11