Deploying AR without telemetry validation is operating blind. The target state after AR enablement is measurable: all spine links should be within 5% utilization of each other during collective workloads, and all-reduce bandwidth should meet or exceed the CloudAI acceptance threshold.

NetQ per-link utilization queries. The primary fabric-wide validation tool is NetQ (covered in depth in Ch28). Use it to query per-port utilization across all spine links simultaneously:

# On NetQ server - check spine link utilization balance
netq show interfaces type swp node spine-01,spine-02,spine-03,spine-04 | grep -E "(swp|util)"

Expected output with AR healthy:

Hostname  Interface  Speed   Util%   RX-Pkts         TX-Pkts
spine-01  swp1       400G    61.2%   48,221,003      47,994,221
spine-01  swp2       400G    59.8%   47,103,441      47,002,118
spine-02  swp1       400G    62.1%   49,001,228      48,882,003
spine-02  swp2       400G    58.9%   46,772,009      46,551,220

The utilization spread between 58.9% and 62.1% is 3.2% - well within the 5% target.

WJH drop detection for AR misconfiguration. What Just Happened (WJH, covered in Ch29) can detect AR-related drops. When AR is misconfigured and per-packet mode is running without SuperNIC reorder, the resulting OOO triggers RoCEv2 NAKs. WJH reports these as RoCE retransmissions:

# On leaf-01 - check WJH for RDMA-related drops
cat /var/log/mellanox/wjh/wjh_dropped_packets.log | grep -i "roce\|rdma\|reorder"

[2026-04-10 14:33:21] ROCE_OOO_NACK: src=10.100.0.4:port49 dst=10.100.0.19:port49
  reason=out_of_order_sequence qp=0x3a8 psn_expected=1042 psn_received=1047
  action=nak_sent count=1,228 last_seen=14:33:21.441

More than 100 ROCE_OOO_NACK entries per minute is a signal that per-packet AR is running without functioning SuperNIC reorder.

Grafana dashboard for AR balance. A practical AR validation dashboard monitors ifHCOutOctets per spine port. With AR working correctly, all spine ports trend together during training jobs. A diverging port (one trending high, others low) indicates AR failure or a misconfigured flowlet timer.

Key Grafana panels for AR validation:

• ifHCOutOctets rate per-port, all spines, 1-minute resolution: should stay within 5% spread
• ar_redirect_count gauge per switch: should show non-zero values when AR is active
• roce_nack_count per DGX node: should be near-zero with AR + reorder functioning

CloudAI benchmark as the definitive AR test. Run the benchmark with AR disabled, then with AR enabled and compare:

# On dgx-node-01 - run CloudAI allreduce benchmark
python3 /opt/nvidia/cloudai/run_benchmark.py \
  --test allreduce \
  --num_gpus 512 \
  --iterations 100 \
  --output /tmp/ar_benchmark_results.json

# Expected: AR enabled result
# allreduce_bw_gbps: 48.3  (pass: >= 46.5 GB/s per Ch25 criteria)
# allreduce_bw_gbps: 36.1  (fail: hash ECMP baseline without AR)

The benchmark result is the definitive validation artifact. Document it in the fabric commissioning record alongside the nv show router adaptive-routing detail output showing Load Std Deviation < 5%.