Skip to content

Chapter 6: Efficient Load Balancing -- Preventing Congestion Before It Starts · Part 8 of 10

Act 7 -- Flow pinning: when you want determinism

Per-packet spraying is not always the right choice. Some workloads benefit from keeping flows on a fixed path.

Flow pinning assigns flows to output paths in a configured pattern -- typically round-robin based on flow characteristics (source IP, QP number) at the time the flow is established. Once assigned, the flow stays on that path for its lifetime.

When is flow pinning preferable to per-packet spraying?

Ordered protocols without reorder support: Some storage protocols and older RDMA implementations cannot handle out-of-order packet delivery even briefly. If reorder buffers are not available (non-RSHP NICs), flow pinning ensures ordered delivery by keeping all packets on the same path.

Debugging and performance analysis: When investigating a performance problem, per-packet spraying makes it difficult to attribute counters to specific flows (because flow packets are spread across multiple paths). Flow pinning makes each flow traceable through the fabric. Production clusters sometimes switch to flow pinning temporarily during deep diagnostic investigations.

Latency-sensitive inference workloads: Inference requests are typically small and latency-sensitive. The per-packet reordering latency introduced by RSHP (up to the reorder tolerance window) can add measurable latency to small requests. For inference, flow pinning with good initial distribution (round-robin assignment) is often preferable to per-packet spraying.

# Leaf switch terminal
# Configure flow pinning with round-robin assignment
nv set interface swp1-32 qos load-balance flow-pinning enable on
nv set interface swp1-32 qos load-balance flow-pinning mode round-robin
nv config apply