The Spectrum-X software stack is designed as a coherent pipeline from switch configuration through packet forwarding through telemetry collection through pre-production validation. Understanding the component interaction order matters operationally: misconfiguring any layer invalidates the behaviour of all downstream layers.
Cumulus Linux 5.x runs on every SN5600 and SN4600C switch as the Network Operating System. Unlike SONiC (which uses an abstraction layer database) or Arista EOS (which is a proprietary NOS), Cumulus is Linux-native: the forwarding table is programmed via kernel netlink APIs, full bash shell access is available, and the FRR (Free Range Routing) suite handles all dynamic routing protocols including BGP, OSPF, and BFD. This means that any Linux tool — tcpdump, iperf3, journalctl, systemd — works directly on the switch CPU.
NVUE (NVIDIA Unified CLI) is the declarative management API layer introduced in Cumulus Linux 5.0. It replaces direct editing of /etc/network/interfaces and /etc/frr/frr.conf with a structured command hierarchy. Configuration is entered as nv set <object> <parameter> <value>, staged in a candidate configuration, then applied atomically with nv config apply. The underlying representation is a JSON document — this means NVUE configuration is serialisable, diffable, and suitable for GitOps workflows. Rollback is nv config history followed by nv config rollback <revision>. The two-phase commit (set → apply) prevents partial configuration states from reaching the forwarding plane.
DOCA SDK on the host side is a set of libraries and runtime daemons that application code links against to access BF3 hardware functions. Applications compiled with DOCA run on the host server Linux, not on the BF3 ARM cores — they communicate with the BF3 via a PCIe messaging channel. This architecture means DOCA applications survive host reboots without manual BF3 restart, and BF3 firmware updates do not require host downtime. Key DOCA applications deployed in Spectrum-X clusters: DOCA Flow for eSwitch packet processing rules, DOCA Compress for checkpoint-to-storage offload, and DOCA App Shield for host integrity monitoring.
NetQ is the fabric-wide telemetry platform. Unlike SNMP, which requires a poller to issue GET requests and receives counter snapshots, NetQ uses a push-based event model: every switch and host runs a lightweight NetQ agent that emits state-change events and time-series metrics to a central NetQ server. The NetQ server stores a queryable time-series database. Queries like "show me all interfaces that had more than 100 drops in the last 10 minutes" execute against stored events rather than requiring a new polling cycle. NetQ also integrates with WJH: drop events captured by the Spectrum-4 ASIC propagate through the NetQ agent to the central database.
NVIDIA Air is the digital twin simulation platform. You export your NVUE topology configuration (switch counts, link connections, IP addressing), import it into Air, and run test traffic against a simulated fabric before deploying the same configuration to production. Air is particularly valuable for BGP policy changes and PFC/ECN threshold tuning — both of which are difficult to test safely in production. Air also supports CI/CD integration via API: a GitOps pipeline can automatically provision an Air simulation, run a smoke test, and block the merge if the simulation fails.
# Apply a new NVUE configuration (two-phase commit)
leaf-01# nv set interface swp1-32 link mtu 9216
leaf-01# nv set interface swp1-32 link speed 400G
leaf-01# nv config diff
--- /etc/nvue.d/startup.yaml
+++ candidate
@@ -10,6 +10,8 @@
swp1:
+ link:
+ mtu: 9216
+ speed: 400G
# ... (32 interfaces shown)
leaf-01# nv config apply
Config applied successfully.
# Verify rollback history
leaf-01# nv config history
Rev Date User Message
1 2024-03-15 09:00 admin Initial baseline
2 2024-03-15 14:22 admin MTU and speed set
3 2024-03-15 15:01 admin (current)
# Query NetQ for recent events on this switch
netq-server$ netq show events level error between now and 30m
Matching events records:
Hostname Message Type Severity Message Time
leaf-01 link error swp29: link down (no signal) 14:58:03