Skip to content

Chapter 18: OOB and Management Network

HPC Networking FoundationsAdvanced60 min read

You have just watched a training job die. The compute fabric is dark -- no ping, no SSH, no NCCL traffic. You need to find out what happened, power-cycle a hung node, check switch console output, and get things moving again. There is one problem: the compute fabric is what you normally use to reach everything.

This is exactly why the out-of-band management network exists. It is a completely separate physical network -- separate switches, separate cables, separate IP ranges -- that gives you a path to every device in the cluster regardless of what the compute or storage fabric is doing. When everything else breaks, the OOB network is how you fix it.

Chapter 1 introduced the concept and described the power-on sequence. This chapter goes inside the machinery: how the BMC works at the silicon level, what IPMI and Redfish actually exchange, how the switch management port differs from its data ports, how UFM reaches switches over the OOB network, and how the DGX B200's BlueField-3 changes the management architecture compared to H100. If you have ever wondered why you need a dedicated 1GbE management NIC when the server already has ten 100GbE ports, this chapter answers that question in full.