Skip to content

Chapter 18: OOB and Management Network · Part 11 of 11

What you now know

  • The BMC is an independent ARM SoC (AST2600) that runs even when the host is powered off. It has its own 1GbE port, its own OS, and its own communication channels to the host (I2C, LPC, virtual NIC). It is the only way to control hardware when the host OS is down.
  • IPMI messages carry a NetFn + Command opcode structure over UDP port 623. Use ipmitool with -I lanplus (IPMI 2.0) for all OOB operations. The KCS path (ipmitool without -H) reaches the local BMC in-band via kernel interface.
  • Redfish is the REST successor to IPMI. Use it for firmware updates and modern orchestration integrations. The resource tree (/redfish/v1/) is self-describing.
  • The OOB switch management port (eth0 on Cumulus, mgmt0 on ONYX) runs in a separate VRF (vrf mgmt) isolated from the data plane forwarding table. It remains reachable even if the data plane is completely misconfigured.
  • UFM uses two separate communication paths: IB MADs over the InfiniBand fabric for topology discovery, and SSH over the OOB network for configuration and firmware updates.
  • The DGX B200 BlueField-3 is a full ARM Linux computer embedded in the NIC. It runs the NVMe-oF initiator on its own ARM cores. Use rshim to push BFB OS images, and SSH directly into the BF3's ARM Linux for storage-layer diagnostics.
  • OOB security means physical separation, changed default credentials, cipher suite 0 disabled, and no route from compute fabric to BMC subnet.

Continue to Chapter 19 -- IP Addressing and Planning ->