As a network engineer who is used to in-band management, the distinction between switch management port and data ports matters here in ways that differ from enterprise switches.
The ONYX and Cumulus Linux switches have a dedicated management Ethernet port -- typically
labelled mgmt or eth0 on the physical chassis. This is not a front-panel data port.
It is a rear or side-panel RJ45 that connects to the OOB switch. The key properties:
Separate routing table: The management port on Cumulus Linux runs in the VRF mgmt.
This is a Linux VRF (Virtual Routing and Forwarding) instance -- a separate routing table
that is completely isolated from the main forwarding table used by the data plane ports.
Traffic sourced from or destined for eth0 uses the mgmt VRF routes, not the main FIB.
# On a Cumulus Linux switch: check the management VRF routing table
ip route show vrf mgmt
# SSH into the switch uses mgmt VRF automatically if you SSH to eth0 IP
# From the switch, to ping something using mgmt VRF explicitly:
ip vrf exec mgmt ping 10.0.2.1
# Management services (sshd, snmpd, etc.) listen only on mgmt VRF interface
# unless explicitly configured otherwise
This separation has a critical implication: if the compute fabric routing table is wrong or corrupted, it does not affect the switch management reachability. You can always SSH into the switch via eth0 even if all 64 data ports are misconfigured. This is why you always configure the management port first and verify SSH connectivity before touching data plane configuration.
Bandwidth: The management port is 1 Gbps, not 100/400 Gbps. It is not in the data
plane ASIC -- it connects to a separate management CPU (typically an ARM core) that runs
the switch OS. Heavy traffic on the management port (for example, a large scp of switch
logs) does not affect forwarding performance. Conversely, a data plane ASIC storm (like
a forwarding table corruption) does not crash the management CPU.
On ONYX switches: The management interface is reached via mgmt0 rather than eth0.
ONYX's IB/Ethernet hybrid architecture has a dedicated management CPU with its own Linux
instance. The ONYX CLI itself runs on this management CPU.
# From UFM or management workstation to an ONYX switch
ssh admin@10.0.2.10
# On the ONYX switch, the management interface config
show interfaces mgmt0
show ip interface mgmt0
# Check management routing table (separate from IB subnet routing)
show ip route vrf mgmt