Scenario A: DCQCN not triggering

Symptom: Single-flow ib_write_bw OK. Multi-flow degrades to ~12 GB/s per flow. tx_ecn_marked_pkts=0 despite congestion.

Root cause: ECN min_threshold set above TC3 buffer capacity. Queue fills to physical limit and PFC fires before ECN marks. DCQCN loop never fires.

Fix:

leaf-01# nv set qos ecn profile roce min-threshold 500000
leaf-01# nv set qos ecn profile roce max-threshold 1500000
leaf-01# nv config apply

Scenario B: PFC deadlock (watchdog fires)

Symptom: Throughput drops to zero within 200ms. Watchdog counter increments.

Root cause: ECN not configured. PFC fires continuously. CNP stuck in paused queue. DCQCN loop broken. Deadlock.

Fix: Apply ECN configuration. ECN prevents queue fill-up, eliminating continuous PFC.

Scenario C: Low ib_write_bw despite healthy links

Symptom: BW ~12 GB/s instead of 46 GB/s. QP transitions without error.

Root cause: MTU mismatch ”” Ethernet MTU is 1500 bytes. IB negotiates to 1024-byte path MTU.

Fix:

dgx-01:~$ ip link set eth0 mtu 9000
leaf-01# nv set interface swp1-32 link mtu 9000
leaf-01# nv config apply

Scenario D: CNP storm

Symptom: tx_ecn_marked_pkts extremely high (millions/second). BW oscillates 30”“46 GB/s every 2”“3 seconds. Classic sawtooth pattern.

Root cause: ECN min_threshold too low (e.g., 4 KB). Every micro-burst triggers marking on every packet. DCQCN over-reacts, rate-limits to near-zero, traffic drains, repeats.

Fix: Raise min_threshold to 500000 (500 KB).