The NVIDIA CloudAI benchmark is the industry-standard acceptance test for Spectrum-X deployments.
What CloudAI measures:
| Test | Metric | Pass threshold (per port) |
|---|---|---|
| All-reduce bandwidth | Aggregate BW across all GPUs | ≥ 380 GB/s (8-GPU DGX) |
| All-to-all bandwidth | Incast stress test | ≥ 360 GB/s |
| Point-to-point | Single-flow line rate | ≥ 46.5 GB/s |
| Latency distribution | p50, p99, p999 | p99 ≤ 2µs, p999 ≤ 10µs |
| Fairness | CV (coefficient of variation) | CV < 5% across flows |
TestAll-reduce bandwidth
MetricAggregate BW across all GPUs
Pass threshold (per port)≥ 380 GB/s (8-GPU DGX)
TestAll-to-all bandwidth
MetricIncast stress test
Pass threshold (per port)≥ 360 GB/s
TestPoint-to-point
MetricSingle-flow line rate
Pass threshold (per port)≥ 46.5 GB/s
TestLatency distribution
Metricp50, p99, p999
Pass threshold (per port)p99 ≤ 2µs, p999 ≤ 10µs
TestFairness
MetricCV (coefficient of variation)
Pass threshold (per port)CV < 5% across flows
Running the benchmark:
dgx-01:~$ cloudai --mode run \
--system-config configs/spectrum-x-8node.toml \
--test-scenario configs/tests/nccl-all-reduce.toml
[cloudai] All-reduce BW: 382.1 GB/s avg (CV=0.6%) ”” PASS
[cloudai] Baseline stored: /results/cloudai/baseline-2024-01-15.json
Comparison against baseline:
dgx-01:~$ cloudai --mode compare \
--baseline /results/cloudai/baseline-2024-01-15.json \
--current /results/cloudai/run-2024-01-22.json
[cloudai] All-reduce BW: 382.1 → 383.4 GB/s (+0.3%) ”” OK
[cloudai] p99 latency: 1.21 → 1.18 µs (-2.5%) ”” OK
[cloudai] Overall: NO REGRESSION DETECTED