Skip to content

Chapter 11: Monitoring, Telemetry, and Observability · Part 7 of 9

Act 6 -- Cross-layer correlation

A single physical event -- a DAC cable with a marginal connector on leaf-3 port 22 -- produces a cascade of observable events across four separate monitoring layers. Understanding the cascade is how you attribute root cause in under 10 minutes instead of 2 hours.

Cross-Layer Correlation Timeline

A single marginal DAC cable triggers a cascade across four monitoring layers. Click any event to see its detail. The layer that anomalies first is the root cause layer.

T+0 minT+22 min
Physical
IB Fabric (UFM)
NCCL
GPU (DCGM)
Training
Social
T+0m
T+5m
T+10m
T+15m
T+20m
IB Fabric (UFM)T+3m
WARNING

IBSymbolErrorRising alert fires

Prometheus rule: rate(symbol_errors[10m]) > 0.17 for 5m → alert fires. PagerDuty notification sent. Engineer investigates during business hours. Link is still Active — no job impact yet.

Tool: Prometheus alert + Grafana
info
warning
critical
With proactive monitoring: action at T+3m. Without: T+20m.

Here is the cascade in order, with the timestamp offset from the initial event:

T+0s -- Physical layer: The marginal connector introduces signal integrity noise. The DAC's DSP starts seeing elevated pre-FEC bit errors. No counter has incremented yet.

T+15s -- IB layer: SymbolErrors begins incrementing on both ends of the link (DGX-07 HCA port 1, and leaf-3 port 22). UFM detects this as a counter delta on the next poll cycle. No alarm has fired yet.

T+3m -- UFM alert: IBSymbolErrorRising alert fires. You get a PagerDuty notification. The link is still Active.

T+7m -- NCCL layer: The IB link's increasing error rate forces retransmissions at the transport layer. AllReduce busbw measured by nccl-tests drops from 146 GB/s to 89 GB/s. NCCL logs show increased timeout_count. The job is still running but slowing.

T+12m -- GPU layer: DCGM shows DCGM_FI_DEV_GPU_UTIL on DGX-07 dropping from 97% to 71%. The GPU is spending more time waiting at the AllReduce barrier. GPUUtilDrop alert fires.

T+18m -- Training logs: The training framework (PyTorch DDP / Megatron) logs a NCCL timeout or Watchdog abort. The job may checkpoint and restart, or may fail entirely.

T+20m -- ML engineer's Slack message: "Training seems slow / crashed."

With a calibrated monitoring stack, you are already investigating at T+3m. Without it, you start at T+20m.

Correlating timestamps across streams

The critical skill is timestamp alignment across four systems that may have different clock sources:

LayerData sourceTimestamp source
IB fabricUFM event APIUFM server clock
GPU / NVLinkDCGM -> PrometheusDGX host clock (NTP)
NCCLLog files on DGXDGX host clock (NTP)
TrainingFramework logsDGX host clock (NTP)

All DGX hosts and UFM server must be NTP-synchronised to the same time source (preferably PTP-disciplined for sub-millisecond accuracy). A clock skew of 5 seconds between the UFM server and a DGX host will make a causal chain look like two independent events.

Practical correlation workflow:

  1. In Grafana: set a time window starting 5 minutes before the reported symptom
  2. Stack the panels: GPU util (DCGM), NVLink BW (DCGM), UFM alarm state, NCCL busbw (if logged to Prometheus via a custom exporter)
  3. Look for the earliest anomaly -- that is your root cause layer
  4. If UFM SymbolErrors leads GPU util drop by 7+ minutes, the fault is physical layer (cable/optics)
  5. If GPU util drops first and UFM shows no errors, the fault is in the compute path (NCCL misconfiguration, NVLink degradation, or CPU-side bottleneck)