Skip to content

BGP-EVPN Multi-Tenancy on Spectrum-X · Part 3 of 8

Act 3 - BGP-EVPN Configuration on Spectrum-X (NVUE Full Walkthrough)

A production two-tenant Spectrum-X fabric configuration proceeds in four logical phases: underlay BGP, VXLAN interface creation, VRF and bridge membership, and BGP-EVPN address-family configuration with route-target assignment. Each phase builds on the previous, and errors introduced in any phase will manifest as routing failures in the next.

Phase 1: Underlay BGP establishes IP reachability between all VTEP loopbacks. On Spectrum-X this is standard eBGP with each leaf in its own AS:

cumulus@leaf-01:~$ nv set router bgp autonomous-system 65001
cumulus@leaf-01:~$ nv set router bgp router-id 10.0.0.1
cumulus@leaf-01:~$ nv set vrf default router bgp neighbor swp1 remote-as 65100
cumulus@leaf-01:~$ nv set vrf default router bgp neighbor swp2 remote-as 65100
cumulus@leaf-01:~$ nv set vrf default router bgp address-family ipv4-unicast network 10.0.0.1/32
cumulus@leaf-01:~$ nv config apply

Phase 2: VXLAN interfaces and bridge membership. Each VNI requires a VXLAN interface and membership in the correct bridge and VLAN:

# Create VXLAN interfaces for tenant VNIs
cumulus@leaf-01:~$ nv set interface vxlan100 vxlan id 100
cumulus@leaf-01:~$ nv set interface vxlan100 vxlan local-tunnelip 10.0.0.1
cumulus@leaf-01:~$ nv set interface vxlan200 vxlan id 200
cumulus@leaf-01:~$ nv set interface vxlan200 vxlan local-tunnelip 10.0.0.1

# Add to bridge and VLAN membership
cumulus@leaf-01:~$ nv set bridge domain br_default vlan 10 vni 100
cumulus@leaf-01:~$ nv set bridge domain br_default vlan 20 vni 200

# Add host-facing ports to VLANs
cumulus@leaf-01:~$ nv set interface swp3 bridge domain br_default access 10
cumulus@leaf-01:~$ nv set interface swp4 bridge domain br_default access 20

Phase 3: VRF creation and L3 VNI assignment for symmetric IRB:

# Tenant VRFs
cumulus@leaf-01:~$ nv set vrf TENANT1 vni 1000
cumulus@leaf-01:~$ nv set vrf TENANT2 vni 2000

# SVI interfaces for routing
cumulus@leaf-01:~$ nv set interface vlan10 ip address 10.10.1.1/24
cumulus@leaf-01:~$ nv set interface vlan10 ip vrr address 10.10.1.254/24
cumulus@leaf-01:~$ nv set interface vlan10 vrf TENANT1
cumulus@leaf-01:~$ nv set interface vlan20 ip address 10.20.1.1/24
cumulus@leaf-01:~$ nv set interface vlan20 ip vrr address 10.20.1.254/24
cumulus@leaf-01:~$ nv set interface vlan20 vrf TENANT2

Phase 4: BGP-EVPN address family and route-target configuration. This is the most operationally critical phase - route-target mismatches are the primary cause of tenant isolation failures:

# Enable EVPN address family
cumulus@leaf-01:~$ nv set vrf default router bgp address-family l2vpn-evpn enable on
cumulus@leaf-01:~$ nv set vrf default router bgp neighbor swp1 address-family l2vpn-evpn enable on
cumulus@leaf-01:~$ nv set vrf default router bgp neighbor swp2 address-family l2vpn-evpn enable on

# Advertise all VNIs
cumulus@leaf-01:~$ nv set evpn enable on
cumulus@leaf-01:~$ nv set evpn multihoming enable on

# Per-VRF route distinguisher and route targets
cumulus@leaf-01:~$ nv set vrf TENANT1 router bgp autonomous-system 65001
cumulus@leaf-01:~$ nv set vrf TENANT1 router bgp rd 65001:100
cumulus@leaf-01:~$ nv set vrf TENANT1 router bgp route-export to-evpn route-target 65000:100
cumulus@leaf-01:~$ nv set vrf TENANT1 router bgp route-import from-evpn route-target 65000:100

cumulus@leaf-01:~$ nv set vrf TENANT2 router bgp autonomous-system 65001
cumulus@leaf-01:~$ nv set vrf TENANT2 router bgp rd 65001:200
cumulus@leaf-01:~$ nv set vrf TENANT2 router bgp route-export to-evpn route-target 65000:200
cumulus@leaf-01:~$ nv set vrf TENANT2 router bgp route-import from-evpn route-target 65000:200

cumulus@leaf-01:~$ nv config apply

Verification - healthy two-tenant fabric:

cumulus@leaf-01:~$ nv show evpn
EVPN Status: enabled
VNIs configured: 4 (2 L2, 2 L3)
BGP sessions with EVPN: 2

cumulus@leaf-01:~$ net show bgp evpn route
BGP table version is 24, local router ID is 10.0.0.1
Status codes: s suppressed, d damped, h history, * valid, > best
Origin codes: i - IGP, e - EGP, ? - incomplete

   Network          Next Hop        In Sid/Label MED LocPrf Weight Path
Route Distinguisher: 65001:100
*> [2]:[0]:[48]:[aa:bb:cc:dd:ee:01]
                    10.0.0.1                           32768 ?
*> [5]:[0]:[24]:[10.10.1.0]
                    10.0.0.1                           32768 ?
Route Distinguisher: 65002:100
*  [2]:[0]:[48]:[aa:bb:cc:dd:ee:02]
                    10.0.0.2                               0 65100 65002 ?
*> [5]:[0]:[24]:[10.10.2.0]
                    10.0.0.2                               0 65100 65002 ?
Route Distinguisher: 65001:200
*> [5]:[0]:[24]:[10.20.1.0]
                    10.0.0.1                           32768 ?
Route Distinguisher: 65002:200
*  [5]:[0]:[24]:[10.20.2.0]
                    10.0.0.2                               0 65100 65002 ?
STEP-BY-STEP CONFIG BUILDER
EVPN Deployment Walkthrough
Phase 1 - Underlay BGP
nv set router bgp autonomous-system 65001
nv set router bgp router-id 10.0.0.1
Leaf loopback becomes reachable fabric-wide.
Underlay BGP
Pending
VXLAN
Pending
Symmetric IRB
Pending
EVPN
Pending
Tenant Isolation
Pending

Notice that the Type-5 routes for TENANT1 (10.10.x.0/24) appear only in the TENANT1 route distinguisher space, and TENANT2 routes (10.20.x.0/24) appear only in their own space. If you ever see a TENANT1 prefix inside TENANT2's RD space, a route-target misconfiguration is importing cross-tenant routes - this is exactly the failure condition Lab 20 explores.