Now the fabric. The purpose of this gate is to prove that every optical path is sound and every cable is in the port the design says it is in — before any protocol behaviour can disguise a physical fault as something more interesting.

Rate and error state. Every compute link should come up at its designed rate with no negotiation surprises. On an HGX B300 node that is eight 800 Gb/s ports; a link that trained at half rate is a bad optic, a bad cassette, or a port in the wrong breakout mode, and it is trivial to find now and painful to find later.

The rail audit is the single highest-value check in the whole bring-up. Rail-optimised topology means GPU n on every node attaches to leaf n. One crossed pair of cables does not break anything — traffic still flows — but it silently converts a leaf-local collective into a spine-crossing one for two nodes, and it surfaces four weeks later as a scalable unit that runs 4 % slow for reasons nobody can explain. Verify it mechanically with LLDP, node by node, against the intended map.

# Local view: which HCA is which netdev, and is it at rate
$ ibdev2netdev -v
mlx5_0 port 1 ==> ens1f0np0 (Up)  800 Gb/s
mlx5_1 port 1 ==> ens2f0np0 (Up)  800 Gb/s
...
mlx5_7 port 1 ==> ens8f0np0 (Up)  400 Gb/s     # <-- trained low, investigate

# Rail audit: every node's rail N must land on leaf N
$ pdsh -g compute 'lldpctl -f keyvalue | grep -E "port.descr|chassis.name"' \
  | ./scripts/rail-map-check.py --expect topology/rail-map.yaml
  cn-0041 rail3 -> leaf-03 swp17   OK
  cn-0042 rail3 -> leaf-04 swp17   MISMATCH (expected leaf-03)
  cn-0042 rail4 -> leaf-03 swp18   MISMATCH (expected leaf-04)
  2 mismatches -- one crossed pair at cn-0042

# Optical health before soak: read module temperature and receive power
$ pdsh -g compute 'mlxlink -d mlx5_0 -m -c | grep -E "Temperature|Rx Power"'

The 24-hour soak. Bring every link up, leave the fabric idle, and read error counters at the start and end of a full day. The acceptance standard is zero growth, not "low". Symbol errors, CRC errors and link-down events on an idle fabric are the fabric telling you about a marginal optic or a dirty connector while it is still cheap to fix. On InfiniBand this is a clean ibdiagnet pass with no counter deltas; on Ethernet it is interface counters and FEC statistics on both ends of every link — remembering, as always, that a link has two ends and only one of them may be reporting.

Exit criteria for G3: all compute links at design rate, rail map verified with zero mismatches, and a 24-hour error soak with no counter growth and no flaps.