ProDeploy — Learn the fabric that keeps AI clusters alive

← Labs ← Chapter 5: PFC & ECNLab 0B - Read RoCE lossless counters

Desktop recommendedChecking session

Lab 0B - Read RoCE lossless counters

beginner

Elapsed

00:00

Scenario

This is a healthy-link counter-reading drill. Why this precursor lab exists: In incident labs, you will see PFC pauses, ECN marks, drops, and bandwidth changes under pressure. If those counters are not familiar yet, troubleshooting becomes guesswork. Topology: DGX Node 01 mlx5_0 / eth0 -> leaf-rail0 swp1 -> remote peer 192.168.100.2 Known-good expectation for this drill: RoCE is enabled on swp1 PFC is enabled on switch priority 3 ECN marking is enabled for the RoCE traffic class A short ib_write_bw probe should run near line-rate with no packet loss Mild ECN marks and a small number of PFC pauses are acceptable under load Your task: 1. Check the leaf-side RoCE/PFC/ECN profile on swp1 2. Run a short ib_write_bw probe from the DGX 3. Read switch-side PFC counters 4. Read switch-side RoCE counters and confirm no-buffer-discard is zero 5. Read host-side NIC counters and confirm ECN marks may be present, but drops stay at zero

Conditions

RoCE profile checked on swp1
ib_write_bw probe completed
Switch PFC counters read
Switch RoCE counters read
Host NIC counters read

Need Help?

Open the step-by-step command guide for this lab.

Topology

Filled green circle = Active

Unfilled red circle = Down

Solid line = 400G link

Dashed line = degraded

Click a device node to open its terminal session

Preparing lab environment...