Skip to content

Chapter 9: Optics, Cabling, and the Physical Layer · Part 7 of 13

Act 6 -- Form factors: OSFP, QSFP-DD, and the port compatibility map

Building the model…

An 800G OSFP transceiver opened up: eight electrical lanes from the host, the DSP, the lasers and the MPO-16 connector. The eye diagram below shows why PAM4 needs the DSP.

A transceiver form factor defines the mechanical and electrical interface between the pluggable module and the switch port. Not all ports accept all transceivers. Getting this wrong before you order hardware means transceivers that physically cannot be installed.

Transceiver form factors — select to compare

Form factor determines which cage a module fits in and which devices are compatible

400GOSFP 400GUsed in DGX H100
Lane configuration
8 × 50G
Modulation
PAM-4
Module power
15 W
Used in
DGX H100 NIC ports, SN5600 leaf switch ports
Module power draw15 W per port
0 W64-port switch total: 960 W in optics alone
Every DGX H100 NIC port is OSFP. Every SN5600 switch port is OSFP. No backward compat — purpose-built for higher power headroom. The OSFP cage is physically larger than QSFP-DD.
Backward compatibility (this cage accepts)

No backward compatibility — purpose-built cage only accepts OSFP 400G modules.

DGX H100 port identity: Every NIC port on a DGX H100 is OSFP. The SN5600 leaf switch also uses OSFP ports. This is why the same cable connects server to switch for both InfiniBand and RoCEv2 modes — the form factor is identical; only the firmware mode differs.

The form factor hierarchy

QSFP28 supports 100G using four lanes at 25G each (NRZ modulation). This was the dominant form factor through the 100G era. A QSFP28 port is electrically identical to a QSFP+ port (40G) -- full backward compatibility.

QSFP56 supports 200G using four lanes at 50G each (PAM-4). Some 200G switch deployments use this. Backward compatible with QSFP28 and QSFP+.

QSFP-DD (Double Density) supports 400G or 800G using eight lanes. The extra four lanes (vs QSFP56) come from a second row of electrical contacts in the connector -- the "double density" in the name. QSFP-DD is backward compatible with QSFP28, QSFP56, and QSFP+. This backward compatibility is its primary advantage -- a switch bought today with QSFP-DD ports can accept 100G modules from existing inventory while being ready for 400G and 800G upgrades.

OSFP (Octal Small Form-factor Pluggable) was designed specifically for the 800G era and higher. It is physically larger than QSFP-DD and has a higher power budget (roughly 20W vs 12W for QSFP-DD). OSFP is not backward compatible with QSFP form factors. OSFP-XD (OSFP Extended Density) is the 2026-era form factor for 1.6T links.

The DGX H100 port identity

Every DGX H100 node has eight OSFP ports -- one per ConnectX-7 NIC. The switch connecting these nodes (the SN5600 leaf switch in NVIDIA's reference design) also has OSFP ports. This is not a coincidence -- NVIDIA designed the H100 DGX system and the SN5600 to use the same form factor, enabling a single cable type to connect server to switch across all eight rails.

This also means that when NVIDIA says "the same OSFP ports are used for both InfiniBand and Ethernet operation" -- they mean the same physical port, same transceiver, same cable. The mode is set in firmware on the ConnectX-7. This is the physical reality underlying Chapter 0's "same cable for IB and RoCEv2" statement.

The heat sink is a compatibility axis, not a finish

Two OSFP modules can be the same form factor, the same speed, and the same optics, and still not be interchangeable. What separates them is the top of the module.

TopWhat it isWhere it goes
Finned-top (IHS)Integrated heat sink, fins on the moduleSwitches only -- Quantum-2, SN5600
Flat-topNo heat sink on the module; the cage provides itHost adapters only -- ConnectX-7 OSFP
Flat-top RHSRiding heat sink; the cage's sink rides on the moduleDGX H100 Cedar links, and liquid-cooled designs
TopFinned-top (IHS)
What it isIntegrated heat sink, fins on the module
Where it goesSwitches only -- Quantum-2, SN5600
TopFlat-top
What it isNo heat sink on the module; the cage provides it
Where it goesHost adapters only -- ConnectX-7 OSFP
TopFlat-top RHS
What it isRiding heat sink; the cage's sink rides on the module
Where it goesDGX H100 Cedar links, and liquid-cooled designs

A finned module physically will not seat in a host cage that expects a flat top, and a flat-top module in a switch cage has no thermal path. This is one of the more common "the replacement optic does not fit" tickets, and it is invisible on a datasheet line that only lists speed and reach.

The DGX H100's Cedar links are the case worth remembering: they take flat-top RHS parts specifically, and neither the switch part nor the ordinary HCA part is a substitute.

Reading the cable before you touch it

The jacket colour is a real convention, not decoration:

  • Black -- direct attach copper
  • Blue -- multi-mode, short reach
  • Orange -- single-mode, long reach

From an aisle, that is enough to spot a single-mode module in a rack that should be all copper, which is usually the tell for a cable that was replaced in a hurry with whatever was on the shelf.

Transceiver power, by part

The table above gives ranges by form factor. When you are doing the actual arithmetic for a rack, the per-part numbers are what you want:

ClassReachModule power
Passive DAC1-3 munder 0.1 W
ACC (linear active copper)3-5 m1.5 W
AOC~30 munder 12 W
Multi-mode, twin-port 800G50 m15 W
Multi-mode, single-port 400G50 m8 W
Single-mode, twin-port 800G100 m - 2 km17 W
Single-mode, single-port 400G100 m9 W
ClassPassive DAC
Reach1-3 m
Module powerunder 0.1 W
ClassACC (linear active copper)
Reach3-5 m
Module power1.5 W
ClassAOC
Reach~30 m
Module powerunder 12 W
ClassMulti-mode, twin-port 800G
Reach50 m
Module power15 W
ClassMulti-mode, single-port 400G
Reach50 m
Module power8 W
ClassSingle-mode, twin-port 800G
Reach100 m - 2 km
Module power17 W
ClassSingle-mode, single-port 400G
Reach100 m
Module power9 W

The shape that matters: a twin-port 800G module costs roughly twice the power of the single-port 400G part beside it, and copper costs two orders of magnitude less than either. A row of racks re-planned from DAC to optics because someone moved the leaf switch does not just cost money; it changes the cooling load.

Power budgets and thermal consequences

Form factor choice directly determines power consumption per port, which determines switch total power, which determines rack cooling requirements:

Form factorSpeedModule power
QSFP-DD400G3-4W per port
QSFP-DD800G8-10W per port
OSFP400G5-8W per port
OSFP800G12-15W per port
OSFP-XD1.6T20-25W projected
Form factorQSFP-DD
Speed400G
Module power3-4W per port
Form factorQSFP-DD
Speed800G
Module power8-10W per port
Form factorOSFP
Speed400G
Module power5-8W per port
Form factorOSFP
Speed800G
Module power12-15W per port
Form factorOSFP-XD
Speed1.6T
Module power20-25W projected

A 64-port 800G switch with OSFP modules: 64 x 15W = 960W in optics alone. Add ~500W for the switch ASIC on a 51.2 Tbps chip, and the total switch power approaches 1,500W -- from a device roughly the width of a pizza box.

Eight such switches in a DGX BasePOD compute fabric: 12,000W from switching alone. This is before accounting for the 32 DGX nodes, storage switches, management networking, or out-of-band infrastructure. Optics are not a minor thermal contribution. They are a primary input to rack cooling design.


Read aloudPage 7