Dragonfly is the third topology family worth understanding. It is used in Cray Slingshot (HPE), which powers Frontier at Oak Ridge National Laboratory -- the first exascale computer -- and Perlmutter at NERSC. It is not used in NVIDIA DGX deployments, but it dominates the scientific HPC market that runs alongside the AI infrastructure world.
The key insight behind dragonfly is that long-distance links are expensive and cable-limited. At very large scale -- tens of thousands of nodes -- a fat-tree requires huge numbers of inter-switch links, and the inter-group links become cable-routing nightmares in a physical facility.
Dragonfly restructures the hierarchy:
Local group: a small clique of switches, all-to-all connected. Each switch connects to every other switch in its group with high-bandwidth local links. Within the group, any two switches can communicate in 1 hop via direct connection or 2 hops via a peer. These are short cables, easy to route.
Global links: each switch in a group connects to exactly one switch in each other group via a single long-distance link. There is only one inter-group link between any pair of groups, rather than the multiple parallel paths that fat-tree provides between levels.
Result: a 3-level hierarchy (host -> local switch -> global link -> local switch -> host) with dramatically fewer long cables and total links than fat-tree at the same node count.
The tradeoff is that the single global link between groups becomes a critical bottleneck. All traffic between two groups must share that one link. This is manageable when traffic has locality, but for random all-to-all communication -- the worst case for AllReduce -- it creates congestion at the global links.
Dragonfly systems address this with adaptive routing: traffic is dynamically rerouted through intermediate groups rather than taking the direct path if the direct global link is congested. A two-hop global route (source group -> intermediate group -> destination group) uses different global links, spreading the load.
Why dragonfly works for scientific HPC but not AI training: scientific codes with spatial locality run near-neighbour communication predominantly within local groups, where performance is excellent. AllReduce for transformer training generates maximum cross-group traffic, which hits the global link bottleneck hard. Adaptive routing helps but cannot fully compensate. Fat-tree's bisection bandwidth guarantee, which dragonfly cannot match for random traffic, is the decisive advantage.
Frontier runs very large-scale scientific simulations -- climate, fusion energy, physics -- that tolerate the dragonfly traffic pattern well. When these machines run AI training, they run it with careful awareness that the interconnect is not optimised for it.