Everything so far has been selection: which part, which reach, which connector. This Act is the other half -- confirming what is actually installed, from the node, without walking the aisle and without switch access.
Chapter 3 covers link state at the DGX end. This is a layer below that. A link can be up and still be running on the wrong part.
Finding the devices
# Start the Mellanox software tools service
sudo mst start
# Register the cable devices so they can be queried
sudo mst cable add
# Added 2 mellanox cable devices.
sudo mst status
The verbose form is the one worth memorising, because it is the single command that ties the four identities of a port together:
sudo mst status -v
# DEVICE_TYPE MST PCI RDMA NET NUMA
# BlueField3 /dev/mst/mt41692_pciconf0.1 17:00.1 mlx5_1 net-eth5 0
PCI address, RDMA device, kernel netdev, NUMA node -- in one row. Every "which port is this, and is it on the right socket?" investigation starts here.
Reading the module
sudo mlxcables -d mt41692_pciconf0_cable_0
Cable Technology : Copper cable unequalized
Cable Type : Passive copper cable
Vendor : NVIDIA
Serial number : MT2431VB61619
Part number : MCP7Y10-N002
Length [m] : 2 m
Digital Diagnostic Monitoring : NO
Attenuation 5g : 5
Attenuation 7g : 6
Attenuation 12g : 7
Attenuation 25g : 14
Cmis max Power : 0.25 W
Three things to take from that output.
The part number and length are authoritative. This is the answer to "is the cable that got installed the cable that was specified?" -- which, after a rushed replacement, is a question worth asking before you start reading counters. A 3 m part where the design says 2 m is a real finding.
Digital Diagnostic Monitoring : NO is correct here, not a fault. This
is a passive copper cable; there is no optical power to report. DDM is a
property of optical modules. Do not go looking for optical readings on a
DAC, and do not treat their absence as a problem. On an optical module the
same command reports temperature, voltage, and per-lane transmit and receive
power -- which is where you look when a link is up but erroring.
Attenuation climbs with frequency. The four figures are the cable's loss at increasing signalling rates, and the shape is the physics from Act 4: 5 at 5 GHz, 14 at 25 GHz. It is why the passive copper reach table ends where it does, measured on the actual cable in your hand rather than assumed from a datasheet.
Where this fits in the workflow
The honest sequence when a link is suspect:
mst status -v-- confirm which device you are even talking about.mlxcables -d ...-- confirm the installed part matches the design, and read optical power if the module is optical.- Chapter 3's link-state and counter checks at the DGX end.
- The switch end, and the asymmetry between the two.
Steps 1 and 2 take under a minute and settle a class of problem that otherwise consumes an afternoon: the link is fine, the fabric is fine, and somebody installed a 50 m multi-mode part on a run that needed single-mode.