A production two-tenant Spectrum-X fabric configuration proceeds in four logical phases: underlay BGP, VXLAN interface creation, VRF and bridge membership, and BGP-EVPN address-family configuration with route-target assignment. Each phase builds on the previous, and errors introduced in any phase will manifest as routing failures in the next.
Phase 1: Underlay BGP establishes IP reachability between all VTEP loopbacks. On Spectrum-X this is standard eBGP with each leaf in its own AS:
cumulus@leaf-01:~$ nv set router bgp autonomous-system 65001
cumulus@leaf-01:~$ nv set router bgp router-id 10.0.0.1
cumulus@leaf-01:~$ nv set vrf default router bgp neighbor swp1 remote-as 65100
cumulus@leaf-01:~$ nv set vrf default router bgp neighbor swp2 remote-as 65100
cumulus@leaf-01:~$ nv set vrf default router bgp address-family ipv4-unicast network 10.0.0.1/32
cumulus@leaf-01:~$ nv config apply
Phase 2: VXLAN interfaces and bridge membership. Each VNI requires a VXLAN interface and membership in the correct bridge and VLAN:
# Create VXLAN interfaces for tenant VNIs
cumulus@leaf-01:~$ nv set interface vxlan100 vxlan id 100
cumulus@leaf-01:~$ nv set interface vxlan100 vxlan local-tunnelip 10.0.0.1
cumulus@leaf-01:~$ nv set interface vxlan200 vxlan id 200
cumulus@leaf-01:~$ nv set interface vxlan200 vxlan local-tunnelip 10.0.0.1
# Add to bridge and VLAN membership
cumulus@leaf-01:~$ nv set bridge domain br_default vlan 10 vni 100
cumulus@leaf-01:~$ nv set bridge domain br_default vlan 20 vni 200
# Add host-facing ports to VLANs
cumulus@leaf-01:~$ nv set interface swp3 bridge domain br_default access 10
cumulus@leaf-01:~$ nv set interface swp4 bridge domain br_default access 20
Phase 3: VRF creation and L3 VNI assignment for symmetric IRB:
# Tenant VRFs
cumulus@leaf-01:~$ nv set vrf TENANT1 vni 1000
cumulus@leaf-01:~$ nv set vrf TENANT2 vni 2000
# SVI interfaces for routing
cumulus@leaf-01:~$ nv set interface vlan10 ip address 10.10.1.1/24
cumulus@leaf-01:~$ nv set interface vlan10 ip vrr address 10.10.1.254/24
cumulus@leaf-01:~$ nv set interface vlan10 vrf TENANT1
cumulus@leaf-01:~$ nv set interface vlan20 ip address 10.20.1.1/24
cumulus@leaf-01:~$ nv set interface vlan20 ip vrr address 10.20.1.254/24
cumulus@leaf-01:~$ nv set interface vlan20 vrf TENANT2
Phase 4: BGP-EVPN address family and route-target configuration. This is the most operationally critical phase - route-target mismatches are the primary cause of tenant isolation failures:
# Enable EVPN address family
cumulus@leaf-01:~$ nv set vrf default router bgp address-family l2vpn-evpn enable on
cumulus@leaf-01:~$ nv set vrf default router bgp neighbor swp1 address-family l2vpn-evpn enable on
cumulus@leaf-01:~$ nv set vrf default router bgp neighbor swp2 address-family l2vpn-evpn enable on
# Advertise all VNIs
cumulus@leaf-01:~$ nv set evpn enable on
cumulus@leaf-01:~$ nv set evpn multihoming enable on
# Per-VRF route distinguisher and route targets
cumulus@leaf-01:~$ nv set vrf TENANT1 router bgp autonomous-system 65001
cumulus@leaf-01:~$ nv set vrf TENANT1 router bgp rd 65001:100
cumulus@leaf-01:~$ nv set vrf TENANT1 router bgp route-export to-evpn route-target 65000:100
cumulus@leaf-01:~$ nv set vrf TENANT1 router bgp route-import from-evpn route-target 65000:100
cumulus@leaf-01:~$ nv set vrf TENANT2 router bgp autonomous-system 65001
cumulus@leaf-01:~$ nv set vrf TENANT2 router bgp rd 65001:200
cumulus@leaf-01:~$ nv set vrf TENANT2 router bgp route-export to-evpn route-target 65000:200
cumulus@leaf-01:~$ nv set vrf TENANT2 router bgp route-import from-evpn route-target 65000:200
cumulus@leaf-01:~$ nv config apply
Verification - healthy two-tenant fabric:
cumulus@leaf-01:~$ nv show evpn
EVPN Status: enabled
VNIs configured: 4 (2 L2, 2 L3)
BGP sessions with EVPN: 2
cumulus@leaf-01:~$ net show bgp evpn route
BGP table version is 24, local router ID is 10.0.0.1
Status codes: s suppressed, d damped, h history, * valid, > best
Origin codes: i - IGP, e - EGP, ? - incomplete
Network Next Hop In Sid/Label MED LocPrf Weight Path
Route Distinguisher: 65001:100
*> [2]:[0]:[48]:[aa:bb:cc:dd:ee:01]
10.0.0.1 32768 ?
*> [5]:[0]:[24]:[10.10.1.0]
10.0.0.1 32768 ?
Route Distinguisher: 65002:100
* [2]:[0]:[48]:[aa:bb:cc:dd:ee:02]
10.0.0.2 0 65100 65002 ?
*> [5]:[0]:[24]:[10.10.2.0]
10.0.0.2 0 65100 65002 ?
Route Distinguisher: 65001:200
*> [5]:[0]:[24]:[10.20.1.0]
10.0.0.1 32768 ?
Route Distinguisher: 65002:200
* [5]:[0]:[24]:[10.20.2.0]
10.0.0.2 0 65100 65002 ?
nv set router bgp autonomous-system 65001 nv set router bgp router-id 10.0.0.1
Notice that the Type-5 routes for TENANT1 (10.10.x.0/24) appear only in the TENANT1 route distinguisher space, and TENANT2 routes (10.20.x.0/24) appear only in their own space. If you ever see a TENANT1 prefix inside TENANT2's RD space, a route-target misconfiguration is importing cross-tenant routes - this is exactly the failure condition Lab 20 explores.