What bisection bandwidth means

Bisection bandwidth is the minimum bandwidth across any cut that divides the cluster into two equal halves. It is the fundamental measure of whether your fabric can support all-to-all communication patterns -- like AllReduce.

If you bisect a 32-node cluster, you get two groups of 16 nodes. During AllReduce, the nodes in one half need to exchange gradient data with the nodes in the other half. If the links crossing the bisection can carry less bandwidth than the server NICs can generate, those links are the bottleneck -- every AllReduce step will be constrained by how fast data can cross the bisection.

Full bisection bandwidth means the links across any bisection cut can carry as much data as all the servers on either side can generate simultaneously. For a cluster with 16 nodes per half, each with 400G of NIC bandwidth, full bisection requires 16 x 400G = 6.4 Tb/s of bandwidth across the cut. If your spine links can only carry 3.2 Tb/s, you have 2:1 oversubscription at the bisection -- AllReduce will run at half speed during fully saturated training.

The k-ary fat-tree construction

A k-ary fat-tree is defined by one number: k, the switch radix (number of ports per switch). A k-ary fat-tree has three layers -- edge (leaf), aggregate, and core -- with specific port allocation rules:

Each switch has k ports.
Each edge switch connects:   k/2 ports -> servers
                             k/2 ports -> aggregate switches (uplinks)

Each aggregate switch:       k/2 ports -> edge switches (downlinks)
                             k/2 ports -> core switches (uplinks)

Core switches:               All k ports -> aggregate switches

For a k-ary fat-tree:

• Number of pods: k
• Edge switches per pod: k/2
• Aggregate switches per pod: k/2
• Core switches: (k/2)^2
• Servers: k^3/4

Worked example with k=8 (8-port switches):

• 8 pods
• 4 edge switches per pod = 32 edge switches total
• 4 aggregate switches per pod = 32 aggregate switches total
• 16 core switches
• 8^3/4 = 128 servers

Worked example with k=64 (64-port switches -- more realistic for AI):

• 64 pods
• 32 edge switches per pod = 2,048 edge switches total
• 32 aggregate per pod = 2,048 aggregate switches total
• 1,024 core switches
• 64^3/4 = 65,536 servers

The k-ary formula shows why switch radix matters so much: doubling radix from 32 to 64 does not double the cluster capacity -- it multiplies it by 8 (2^3). High-radix switches are disproportionately powerful for building large clusters.

Real hardware: A Spectrum-X SN5600 leaf switch in a data center rack. 64 OSFP ports visible on the front panel. Each port carries 400G. Half the ports face downward toward DGX nodes; half face upward toward spine switches.

![NVIDIA Spectrum-X SN5600 switch front panel -- 64 OSFP 400G ports] Source: NVIDIA networking hardware documentation Image placeholder: Search "NVIDIA Spectrum-X SN5600 datasheet" for the official front-panel image showing the 64-port OSFP configuration.

Leaf-spine as a 2-stage fat-tree

The leaf-spine topology used in DGX deployments is a special case of the k-ary fat-tree with only two stages (edge + aggregate, no core). This is also called a Clos network after the Bell Labs engineer Charles Clos who formalised it in 1953.

In a 2-stage fat-tree:

• Leaf switches connect to DGX nodes (downlinks) and spine switches (uplinks)
• Spine switches only connect to leaf switches -- no servers

The number of spine switches equals the number of uplink ports per leaf switch. If each leaf switch has 32 downlinks (to DGX NICs) and 32 uplinks (to spine), you need 32 spine switches. Adding more spine switches requires changing the leaf switches to have more ports, which means buying different hardware.

This is the BasePOD configuration. It is simple, fast, and fully non-blocking -- every server port has exactly one spine port backing it up. The limitation is scale: to connect more nodes, you eventually need a third layer.