Skip to content

Chapter 19: IP Addressing and Planning · Part 11 of 11

What you now know

  • Every address family in an AI fabric has a purpose. Loopbacks are router identities. /32 host routes are how servers appear in BGP. OOB addresses are physically separate. Storage addresses are on a different physical switch. Each family lives in its own /16 block.
  • The top-level 10.0.0.0/8 partitioning is the most important single decision. OOB = 10.0.x.x, Compute = 10.10.x.x, Storage = 10.20.x.x, Tenants = 10.30.x.x. Non-overlapping by construction and summarisable at pod boundaries.
  • Switch loopbacks are /32 host routes, pod-indexed in a /24 block, with reserved ranges for leaf/spine/super-spine expansion. The loopback is the BGP router-ID, the VTEP IP, and the management polling target.
  • BGP unnumbered (RFC 5549) eliminates all P2P link addressing. Numbered /31 links are only needed for troubleshooting convenience, third-party compatibility, or external peering.
  • VNIs encode pod + tenant + segment in a 24-bit number. VLAN IDs are deduplicated by range across fabrics. VLAN exhaustion (>4094) is a real risk in large multi-tenant deployments and must be planned from day one.
  • BasePOD-to-SuperPOD scaling works by BGP route aggregation. Each pod advertises a summary prefix, not individual /32s. This only works if the pod's address space is designed to be aggregatable from the start.
  • The seven common mistakes all share one root cause: addressing decided ad-hoc without a plan. A one-hour addressing session before racking a single switch prevents a week of debugging after every GPU is running.

Continue to Chapter 20 -- Ultra Ethernet Consortium ->