Every mistake in this list has been made in production. Some caused hours of debugging. Some caused multi-day outages.

Mistake 1: Compute and OOB in overlapping ranges

What happens: You allocate 10.0.0.0/8 for compute and also use 10.0.1.0/24 for BMC addresses. The DGX compute NIC has a route to 10.0.0.0/8 via BGP. When it tries to reach the BMC at 10.0.1.10, the packet goes over the compute fabric instead of the OOB switch. The BMC never receives it because the compute switch has no route to the BMC. Management is unreachable via the path it should be unreachable via, and the engineer cannot figure out why ipmitool hangs.

Fix: OOB and compute must be in entirely separate /16 blocks. No overlap possible by construction. 10.0.x.x = OOB always. 10.10.x.x = compute always.

Mistake 2: /24 server subnets instead of /32 host routes

What happens: You give each DGX-leaf segment a /24 subnet (DGX-01 on 10.10.1.0/24, DGX-02 on 10.10.2.0/24). AllReduce traffic needs to flow between DGX-01 and DGX-02. The leaf has two /24 subnets in the routing table but they are directly connected -- no BGP route is generated because they are connected routes, not BGP-learned routes. Traffic within a DGX works fine. Traffic between DGX nodes fails because no route exists across the fabric.

Fix: Server interfaces as /32. Leaf advertises /32 into BGP. Spines learn /32 and forward. Remote leaves install /32 as BGP route. Works.

Mistake 3: VLAN ID reuse across storage and compute

What happens: VLAN 10 is used for compute traffic on the leaf switches and also VLAN 10 is configured on the storage switch for storage management. When the leaf switch's VLAN 10 trunking port is plugged into the storage switch accidentally (wrong cable in a dense rack), VLAN 10 traffic merges. GPU AllReduce packets flood the storage network. Storage appliances receive millions of unfamiliar multicast packets. NVMe-oF connections reset under the load.

Fix: Dedicate VLAN ranges by fabric. VLANs 1-99: reserved/infrastructure. VLANs 100-199: storage. VLANs 1000-4094: compute/tenants. Never share a VLAN ID across fabrics.

Mistake 4: Not reserving headroom in every /24

What happens: BasePOD 1 is deployed with 8 DGX nodes. Each uses 8 IPs in 10.10.1.0/24. 64 addresses consumed of 254. Six months later, the cluster is expanded to 32 DGX nodes. Now you need 256 addresses in 10.10.1.0/24 -- one more than the /24 can hold. Either the 30th DGX gets an address in a different /24 (breaking the pod-indexed scheme) or you renumber the entire pod.

Fix: Plan for maximum pod size before deployment. A BasePOD is typically 32 DGX nodes maximum. 32 x 8 = 256 compute addresses. Allocate a full /23 (10.10.0.0/23 = 512 addresses) for the compute interfaces of one pod, not a /24. Reserve half for future use.

Mistake 5: BGP router-ID collision

What happens: Two switches in the fabric have the same loopback IP. They both advertise the same /32 into BGP. Each switch sees two equal-cost paths to 10.10.0.1/32. Depending on which switch is queried, traffic that should reach switch A arrives at switch B. SNMP polling for switch A returns switch B's counters. UFM topology shows switches flickering between two apparent locations.

Fix: Automate loopback assignment. Generate loopbacks from the pod-indexed scheme in a spreadsheet and push via Ansible. Never assign by hand without a source-of-truth document.

Mistake 6: VNI assignment without a naming convention

What happens: Tenant 1 gets VNI 100. Tenant 2 gets VNI 200. Later, a temporary debug VNI 150 is created and forgotten. Six months later, a new engineer adds VNI 100 to a second pod and conflicts with a VNI range that extends across both pods. EVPN distributes conflicting MAC-IP bindings. Traffic for one tenant randomly ends up in another tenant's VRF.

Fix: Use the formula from Act 7. Every VNI encodes pod + tenant + segment. Any VNI is immediately decodable. Conflicts are prevented by construction.

Mistake 7: No summary route plan at SuperPOD interconnect

What happens: BasePOD 1 and BasePOD 2 are connected at the super-spine. Neither pod was designed with aggregatable address ranges. BasePOD 1 uses 10.5.1.0/24 for some compute, 10.8.3.0/24 for other compute (assigned ad-hoc). There is no clean summary. The super-spine must carry the full BGP table of both pods -- thousands of /32 host routes. When super-spine-01 reboots, it takes 4 minutes to reconverge because it must re-learn every individual /32 from both pods.

Fix: Design the summarisation into the address plan from day one. Pod N owns 10.10.(N*10).0/23 for compute. The super-spine carries two /23 summaries per pod. Convergence is seconds, not minutes.