Now that you understand why the hardware evolved this way, you can look at the DGX H100 and understand every design decision:
DGX H100 — anatomy and alternatives
Figure: Official NVIDIA DGX H100 system image. Seeing the chassis itself helps ground the anatomy discussion in the physical box operators actually deploy.
Every choice in the DGX design is an answer to a specific constraint:
| Design decision | Why |
|---|---|
| 8 GPUs per chassis | Maximum NVLink bandwidth per GPU within practical power and thermal limits |
| 4 NVSwitch chips | Non-blocking all-to-all between 8 GPUs at 900 GB/s per GPU |
| 1 ConnectX-7 per GPU | No shared uplinks -- each GPU gets its own 400G path to the fabric |
| 8 NICs on 8 different switches | Rail topology -- fault isolation and maximum AllReduce performance |
| 2 Intel Xeon CPUs | Management only -- training is GPU + NIC, CPUs are mostly idle |
| 8x 3.84 TB NVMe U.2 (~30 TB) | Local OS, checkpointing -- training data lives on shared parallel storage (BeeGFS, WEKA) |
The same table, one generation on. An HGX B300 baseboard keeps every structural decision above and changes three numbers. Eight Blackwell Ultra GPUs replace eight H100s, with 288 GB of HBM3e each and about 2.1 TB per board. NVLink 5 gives each GPU 1.8 TB/s to its siblings -- double NVLink 4 -- for 14.4 TB/s across the onboard switch. And the per-GPU fabric port doubles as well: one ConnectX-8 SuperNIC at 800 Gb/s per GPU, which NVIDIA quotes as 1.6 TB/s of networking bandwidth per node against 0.8 TB/s for HGX B200.
That last row is the one that reshapes the network. Rail topology, one NIC per GPU, eight NICs on eight switches -- all unchanged. What changes is that the leaf tier is now built from 800 GbE OSFP cages, the optics move a generation, and a congestion event fills a switch queue in half the time it used to. Chapter 14 works through the platform in detail; Chapter 31 works through what it does to a cluster design.