SR-TE (Traffic Engineering with Segment Routing) is the practical mechanism for the use case described in Act 1: steering different traffic types onto different spine paths.

The AI fabric SR-TE model

In an AI fabric, the headend is typically the leaf switch (ToR switch connected to GPU nodes). The leaf switch classifies traffic by DSCP or source/destination prefix and encapsulates it with a pre-programmed SID list.

GPU node → leaf-01 (headend) → SRv6 encapsulation → path through spine-2/3 → leaf-08 → storage
                                                                ↑
                                          SID list: [spine02::End, spine03::End, leaf08::End.DT4]

The GPU node itself does not need to be SRv6-capable. The leaf switch performs the encapsulation and decapsulation. This is called SR-TE with headend encapsulation.

Configuring an SR-TE policy

An SR-TE policy defines:

• Endpoint: the final destination (the decap point)
• Color: a tag that identifies the intent (e.g., "low-latency path")
• Candidate paths: one or more SID lists, with preference values

# FRR configuration for SR-TE policy on leaf-01:
vtysh
segment-routing
 traffic-eng
  policy CHECKPOINT-STORAGE
   color 100 endpoint 2001:db8:0:leaf08::1
   candidate-path preference 100 explicit segment-list CHECKPOINT-PATH
   !
  !
  segment-list CHECKPOINT-PATH
   index 10 mpls label 0 ipv6 2001:db8:0:spine02::1    # End SID for spine-02
   index 20 mpls label 0 ipv6 2001:db8:0:spine03::1    # End SID for spine-03
   index 30 mpls label 0 ipv6 2001:db8:0:leaf08::100   # End.DT4 SID for storage VRF
  !
 !
!

# Verify the policy is active:
vtysh -c "show sr-te policy"
# Expected:
# Policy CHECKPOINT-STORAGE
#   Endpoint: 2001:db8:0:leaf08::1
#   Color: 100
#   Status: Active
#   Binding SID: 2001:db8:0:leaf01::200   ← auto-assigned
#   Candidate-paths: 1 active

DSCP-to-SR-policy mapping: the integration point

NCCL marks outbound packets with DSCP 26 (RoCE traffic, priority 3). Checkpoint-to-storage traffic is typically unmarked or uses DSCP 10 (AF11). The leaf switch maps incoming DSCP values to SR-TE policies using a route policy:

# FRR route-map for DSCP-based SR-TE steering:
route-map STEER-TO-SRTE permit 10
  match ip dscp 10                         # Match checkpoint traffic (DSCP 10)
  set sr-te color 100                      # Assign to SR-TE policy color 100
!
route-map STEER-TO-SRTE permit 20
  match ip dscp 26                         # Match RoCE training traffic
  # No set action = follow normal ECMP (no SR-TE steering)
!

# Apply to the ingress interface (toward GPU nodes):
interface swp1
  ip policy route-map STEER-TO-SRTE
!

# Verify steering is working under load:
vtysh -c "show sr-te policy detail" | grep -i "packets\|bytes"
# Shows traffic counters per SR-TE policy

The critical property: NCCL requires no changes. NCCL already marks DSCP 26 by default (or configurable via NCCL_NET_OVERHEAD_DSCP). The network layer reads that DSCP and makes the path decision. Application-transparent traffic engineering is one of SR-TE's most operationally valuable properties.

Binding SIDs: simplifying the SID list

For a complex path with 6 SIDs, distributing the full SID list to every headend that needs it is operationally expensive. Binding SIDs (BSIDs) solve this. A BSID is a single SID that the headend pushes onto the packet. When the packet reaches the BSID owner, it expands the BSID into the full SID list.

Without BSID:  leaf-01 → [spine02, spine03, leaf08::DT4]  ← 3 SIDs
With BSID:     leaf-01 → [leaf08::BSID-200]               ← 1 SID
               When packet reaches leaf-08, it sees BSID-200 and steers accordingly

In large AI fabrics with 32+ leaf switches, using BSIDs means you only manage the full SID list in one place (the endpoint leaf). All headends use the BSID. This dramatically simplifies Day-2 operations when paths change.