Skip to content

Liquid Cooling and Heat Rejection · Part 7 of 7

Act 6 — Optional: Rack-Scale Systems and What Comes Next

Building the model…

GB300 NVL72: 18 compute trays and 9 NVLink switch trays joined by a copper spine into one 72-GPU NVLink domain. Pull a tray to see inside.

The material above applies to any direct-to-chip deployment. This section is optional context for teams planning rack-scale Blackwell Ultra or looking one generation ahead. Treat every figure as a starting point to check against the current datasheet — this part of the roadmap moves quickly.

GB300 NVL72 is where liquid cooling stops being a rack option and becomes the rack. Seventy-two Blackwell Ultra GPUs and thirty-six Grace CPUs sit across 18 compute trays; nine NVLink switch trays, two NVSwitch ASICs each, close a fully connected 72-GPU domain at 130 TB/s of aggregate NVLink bandwidth. Eight 33 kW power shelves feed a DC busbar, with the rack budgeted up to 142 kW. Compute trays, switch trays and the power path are all on cold plates; leak detection exists at both tray and rack level. Scale-out leaves the rack on 72 ConnectX-8 SuperNICs — four per compute tray, 800 Gb/s per GPU — into either Quantum-X800 InfiniBand or Spectrum-X Ethernet, with BlueField-3 handling the north–south path and a pair of SN2201 switches carrying out-of-band management.

For the cooling designer, the important shift is that the unit of thermal design is now the rack, not the node. You do not get to under-populate a tray to shed heat, and you do not get to stage the deployment thermally. The rack arrives with a flow requirement and a supply-temperature ceiling, and the room either meets them on day one or the rack does not come up.

Vera Rubin raises every number again. NVIDIA's announced platform pairs the Rubin GPU — 288 GB of HBM4 at up to 22 TB/s, 3.6 TB/s of NVLink 6 per GPU — with the 88-core Vera CPU, and scales out on ConnectX-9 at 1.6 Tb/s per GPU into Spectrum-6 Ethernet switches built with co-packaged optics. The rack-scale form is 72 Rubin GPUs, launched as Vera Rubin NVL72 (earlier roadmap material called it NVL144, counting compute dies rather than GPU packages -- both names refer to the same 72-package rack). Published rack power for the generation sits well above NVL72's envelope, and NVIDIA has signalled a move to 800 VDC distribution to keep busbar currents tractable.

Three planning consequences follow, and they are worth carrying into any facility decision made this year:

  • Design the loop for the rack after next. Pipe diameter and header capacity are structural; pumps and CDUs are replaceable. Sizing headers for today's 142 kW rack and next year's substantially higher one costs very little at construction and is close to impossible to retrofit.
  • Co-packaged optics change the residual air budget in your favour. Moving the optical engine into the switch package removes a large share of the pluggable-transceiver heat that currently rides the air stream — Spectrum-6 is quoted at roughly five times better network power efficiency on that basis. The air load in a Rubin-era rack is a different shape, not simply a bigger one.
  • Expect ΔT to widen, not flow to grow indefinitely. There is a practical ceiling on litres per minute through a rack manifold. The industry's answer to higher rack power is warmer return water and wider ΔT, which is another argument for specifying the loop's temperature band generously now.

None of this changes the operator's day. The loop still has a supply temperature, a flow rate, a filter that loads, and a leak sensor that must be tested. What changes is how little headroom you have when one of them drifts.

Read aloudPage 7