Skip to content

Segment Routing for AI Fabrics · Part 5 of 8

Act 5 — EVPN + SRv6 for Multi-Tenant AI

Chapter 15 covered EVPN-VXLAN for multi-tenancy. EVPN+SRv6 replaces VXLAN with SRv6 as the underlay transport, and the combination is increasingly the preferred design for new GPU cloud infrastructure deployments.

The EVPN-VXLAN problem at scale

EVPN-VXLAN requires each leaf to:

  1. Maintain a VTEP (VXLAN Tunnel Endpoint) for every remote leaf
  2. Encapsulate every tenant frame in a VXLAN header (8 bytes) inside a UDP/IP header (28 bytes)
  3. Perform encapsulation and decapsulation at every leaf-to-leaf boundary

For a 64-leaf SuperPOD, each leaf maintains 63 VTEP tunnels. Under high-availability scenarios, the VTEP table state must be replicated and verified. The 36-byte VXLAN/UDP/IP encapsulation reduces effective MTU for the inner payload.

EVPN+SRv6: the simplification

In EVPN+SRv6, the tenant route (RT-5, IP prefix route) carries an SRv6 VPN SID as its next-hop instead of a VTEP IP address. The SID encodes both the destination (which leaf to reach) and the tenant context (which VRF to deliver into). No VXLAN headers, no VTEPs.

EVPN-VXLAN route announcement:
  RT-5: 10.100.1.0/24
    Next-hop: VTEP 192.168.1.5       ← The VTEP IP of the remote leaf
    VXLAN VNI: 1000                  ← Carried separately

EVPN+SRv6 route announcement:
  RT-5: 10.100.1.0/24
    Next-hop: 2001:db8:0:leaf05::100 ← This IS the SID (End.DT4 in VRF RED)
                                       No VTEP, no VNI — SID encodes everything

When traffic is forwarded to 10.100.1.0/24 from leaf-01, leaf-01 encapsulates it with an IPv6 outer header destined to 2001:db8:0:leaf05::100. The Spectrum-4 ASIC on leaf-05 sees this destination matches its local End.DT4 SID for VRF RED, decapsulates, and performs an IPv4 lookup in VRF RED. No VXLAN header, no VNI lookup — the SID carries it all.

COMPARISON
EVPN-VXLAN vs EVPN+SRv6: Control and Data Plane
Step through control plane events and data plane encapsulation for both approaches.
EVPN-VXLAN
BGP EVPN RT-5 route with VTEP next-hop
BGP UPDATE:
  Route Type: 5 (IP Prefix)
  Route Distinguisher: 65001:100
  Prefix: 10.100.1.0/24
  Next-hop: 192.168.1.5       ← VTEP IP of remote leaf
  VXLAN VNI: 1000             ← Separate attribute
  ESI: 0 (single-homed)
  
# Each leaf must maintain:
# - VTEP table for 63 remote leaves
# - VNI-to-VRF mapping table
# - MAC/ARP table per VNI
Control plane carries VTEP IP + VNI separately. Receiver must maintain VTEP state and correlate VNI at packet arrival.
EVPN+SRv6
BGP EVPN RT-5 route with SRv6 VPN SID
BGP UPDATE:
  Route Type: 5 (IP Prefix)
  Route Distinguisher: 65001:100
  Prefix: 10.100.1.0/24
  Next-hop: 2001:db8:0:leaf08::100  ← SID encodes BOTH
                                       destination AND VRF
  # No separate VNI attribute needed
  # No VTEP table needed
  # SID = End.DT4 (decap + VRF RED lookup)

# Each leaf only needs:
# - Normal IPv6 route to the SID prefix (via IS-IS)
# - Local SID table (own SIDs only)
The SRv6 VPN SID encodes both where to go (leaf-08) and what to do there (VRF RED lookup). No VTEP state, no VNI correlation.
PropertyEVPN-VXLANEVPN+SRv6
Outer encap overhead50 bytes (Eth+IP+UDP+VXLAN)40 bytes (IPv6 only)
VTEP state per leafN-1 VTEP entriesNone
Tenant ID mechanismVNI (separate attribute)Embedded in SID function
Control plane lookups (egress)VTEP → VNI-VRF → FIB (3)SID → VRF FIB (2)
MTU headroom−50 bytes−40 bytes (or 0 w/ uSID)
SR-TE path steeringNot nativeNative (SID list in header)

EVPN+SRv6 configuration walkthrough

# On leaf-01: configure EVPN+SRv6 for Tenant RED
router bgp 65001
  address-family l2vpn evpn
    advertise-all-vni
    segment-routing srv6
     locator LEAF01                   # Use leaf-01's SRv6 locator
  !
!

# Assign End.DT4 SID to VRF RED:
router bgp 65001
  vrf RED
    address-family ipv4 unicast
      segment-routing srv6
       locator LEAF01
       sid-allocation per-vrf         # One SID per VRF (vs per-prefix)
    !
  !
!

# Verify EVPN routes are carrying SRv6 VPN SIDs:
vtysh -c "show bgp l2vpn evpn route type 5"
# Expected:
# Route Distinguisher: 65001:100
#   *> 10.100.1.0/24 via 2001:db8:0:leaf03::100
#      SRv6 VPN SID: 2001:db8:0:leaf03::100 (End.DT4/VRF RED)
#      Path: 65001 65003

Multi-tenant traffic separation with SRv6 + GBP

When combined with GBP (Group-Based Policy, Chapter 27), SRv6 provides both path isolation and security policy enforcement. The SRv6 path carries traffic to the correct VRF; GBP at the egress leaf enforces whether the source EPG (Endpoint Group) is allowed to communicate with the destination EPG. The two mechanisms are complementary and operate at different layers.