Skip to content

Congestion Control Deep Dive · Part 5 of 9

Act 5 — TIMELY: RTT-Gradient CC on Commodity NICs

TIMELY (Transport Improvement through Measurement of Latency with You) was published by Google in SIGCOMM 2015 and is notable for one reason: it achieves RTT-based congestion control on commodity NICs by computing RTT gradient from software timestamps.

The gradient insight

TIMELY does not look at absolute RTT; it looks at the rate of change of RTT — the gradient. The reasoning is that RTT gradient is an earlier signal:

  • DCQCN signal: queue depth crosses min_threshold → ECN mark → 2 RTTs later, sender reduces rate
  • Swift signal: RTT exceeds target_delay → 1 RTT later, sender reduces rate
  • TIMELY signal: RTT is increasing (even before exceeding any threshold) → immediate response
RTT_gradient = (RTT_current - RTT_prev) / measurement_interval

if RTT_gradient >= 0 (RTT is rising):
    rate = rate × (1 - β × normalized_gradient)    # multiplicative decrease
else (RTT is falling):
    rate = rate + δ                                  # additive increase

Where β is the decrease factor (0.8 typical) and δ is a fixed additive increment (typically 5 Mbps). The gradient normalisation prevents overreaction to transient noise.

Why gradient is an earlier signal

Under incast onset, the queue builds from zero. For the first few packets arriving before the queue fills significantly, absolute RTT may still be below any threshold. But the RTT is rising — and TIMELY detects that rise immediately. This gives TIMELY a systematic one-measurement-interval advantage over threshold-based schemes.

The practical tradeoff

TIMELY's RTT measurements come from software timestamps (gettimeofday) rather than NIC hardware timestamps. Software timestamp variance can be 5–20 µs on a busy kernel. For AI fabrics with baseline RTTs of 1–5 µs, software timestamp noise can exceed the signal. This limits TIMELY to higher-RTT environments (10 µs+, across multiple switch hops).

On commodity TCP flows in a data centre with 20+ µs baseline RTT (e.g., between racks in different pods), TIMELY works well. For intra-pod GPU collectives with sub-5 µs RTTs, the noise floor makes TIMELY impractical without hardware timestamp support.

COMPARATIVE MODEL
TIMELY Gradient vs Threshold-Based Algorithms
Shows which algorithm detects the onset of congestion first. Burst starts at t=20µs.
0µs5µs10µs15µs20µsECNSwiftRTT (measured)0+2-1dRTT/dt (gradient)TIMELYSwiftDCQCN0µs20µs40µs60µs80µs100µs
TIMELY reacts
20.0µs
gradient > 0
Swift reacts
46.5µs
RTT > target
DCQCN reacts
33.0µs
ECN + 2×RTT
Gradient advantage: TIMELY detects rising RTT immediately at burst onset (t=20µs). Swift waits until RTT exceeds 5.8µs target. DCQCN waits until queue fills to ECN threshold (500KB → ~3µs extra RTT) then adds 2 more RTTs before the sender responds.