Skip to content

Chapter 19: IP Addressing and Planning · Part 3 of 11

Act 2 -- RFC 1918 Space Partitioning

RFC 1918 defines three private address ranges:

  • 10.0.0.0/8 -- 16 million addresses
  • 172.16.0.0/12 -- 1 million addresses
  • 192.168.0.0/16 -- 65,536 addresses

For AI fabric deployments, 10.0.0.0/8 is the only range large enough. Use the full /8 and partition it cleanly between fabrics, pods, and device types. 172.16.x.x and 192.168.x.x are often already in use in the corporate network you are connecting to -- using them for the AI cluster creates the nightmare scenario where two default routes exist and traffic black-holes.

RFC 1918 Partitioning
10.0.0.0/8 -- Full Allocation Plan
10.0.0.0/8 address space -- each cell = one /16 block (third octet shown)
10.0.0.0/16
10.10.0.0/16
10.20.0.0/16
10.30.0.0/16
10.100.0.0/16
10.200.0.0/16
unallocated
10.10.0.0/16 -- Compute Fabric
Switch loopbacks, server /32 host routes, in-band management. All routes present in BGP fabric. Summarisable per pod as /24.
Subnet Allocations
10.10.0.0/24
Switch loopbacks -- Pod 1 (leaf/spine/super-spine)
10/254
10.10.1.0/24
DGX compute /32s -- Pod 1, Rail 0+1
64/254 (8 DGX x 8 ports)
10.10.2.0/24
DGX compute /32s -- Pod 1, Rail 2+3 (overflow)
0/254 (reserved)
10.10.4.0/24
DGX in-band management (CX7 bond)
8/254
10.10.5.0/24
P2P links (ONLY if using numbered links)
0/254 (unused in unnumbered)
10.10.10.0/24
Switch loopbacks -- Pod 2
0/254 (reserved)
10.10.11.0/24
DGX compute /32s -- Pod 2
0/254 (reserved)
10.10.100.0/24
Super-spine loopbacks
4/254
10.10.128.0/17
RESERVED -- future pods
0/32767

The recommended top-level partitioning for a multi-pod AI deployment:

10.0.0.0/8  --  ENTIRE AI CLUSTER
  |
  +-- 10.0.0.0/16   OOB management (BMC + switch mgmt0)
  |     10.0.1.0/24   DGX BMC addresses (BasePOD 1)
  |     10.0.2.0/24   Switch management addresses (BasePOD 1)
  |     10.0.3.0/24   UFM / monitoring servers
  |     10.0.10.0/24  DGX BMC addresses (BasePOD 2)
  |     ... /24 per pod, sub-allocated
  |
  +-- 10.10.0.0/16  Compute fabric loopbacks + server prefixes
  |     10.10.0.0/24  Switch loopbacks (BasePOD 1)
  |     10.10.1.0/24  Server host prefixes (BasePOD 1, rail 0)
  |     10.10.2.0/24  Server host prefixes (BasePOD 1, rail 1)
  |     10.10.10.0/24 Switch loopbacks (BasePOD 2)
  |     ... /24 per pod
  |
  +-- 10.20.0.0/16  Storage fabric
  |     10.20.1.0/24  DGX storage NICs (BasePOD 1)
  |     10.20.2.0/24  Storage appliance portals (BasePOD 1)
  |     10.20.10.0/24 DGX storage NICs (BasePOD 2)
  |
  +-- 10.30.0.0/16  EVPN tenant VRF subnets
  |     10.30.1.0/24  Tenant 1 GPU network
  |     10.30.2.0/24  Tenant 2 GPU network
  |     ... one /24 per tenant
  |
  +-- 10.100.0.0/16  Reserved for future SuperPOD expansion
  +-- 10.200.0.0/16  Reserved for interconnect to corporate network

This partitioning has three properties a network engineer cares about:

  1. Summarisable: each pod's complete address usage fits in a small number of /16 or /24 aggregates. When you peer a BasePOD BGP fabric with a SuperPOD spine, you advertise 10.10.0.0/16 as a summary, not hundreds of /32 host routes. The corporate firewall policy stays manageable.

  2. Non-overlapping by construction: OOB is always 10.0.x.x. Compute is always 10.10.x.x. Storage is always 10.20.x.x. There is no scenario where a DGX host confuses its BMC address with its GPU compute address because they are in different /16 blocks.

  3. Pod-indexed: the third octet encodes the pod number. Pod 1 = .1.x, pod 2 = .10.x (leaving room within the pod for device type sub-allocation). When you see 10.10.1.45 in a log, you immediately know: compute fabric, BasePOD 1, server address.