The edge is where a cluster stops being a cluster and becomes a network with customers. Two structurally different models are in common use, and the choice determines how failover behaves.
Model A — clustered firewalls behind WAN switches. Two firewall appliances run as a high-availability cluster with a shared identity, synchronised session state, and one active unit holding the routing adjacencies. Each carrier hands off into a small dedicated WAN switch, and each WAN switch connects to the same port on both firewalls, so either unit can serve either carrier. Failover is an appliance-level event: the standby promotes, addresses move, synchronised sessions survive.
Model B — two independent routers, carriers attached directly. Each firewall is a standalone speaker with its own addresses, its own router ID, and its own carrier. There is no cluster, no shared address, and no promotion. Failover is purely a routing event: the failed path's default route is withdrawn, and traffic converges to the other unit. Session state may be synchronised for supported protocols, but synchronisation is not the same thing as forwarding — a packet arriving at the wrong unit may still need to reach the session owner for inspection, and a sync link does not automatically provide that path.
| Clustered (Model A) | Independent (Model B) | |
|---|---|---|
| Extra hardware | Two WAN switches | None |
| Failover mechanism | Appliance promotion + session sync | BGP withdraw and reconvergence |
| Existing sessions on failure | Often preserved | Generally reset |
| Asymmetric-path risk | Low | Must be designed for explicitly |
| Failure modes to test | Cluster split-brain, sync loss | Route policy correctness, asymmetric inspection |
Neither is wrong. Model A buys session continuity with hardware and cluster complexity; Model B buys simplicity and removes a shared-fate cluster, and pays for it with connection resets on failover and a genuine design problem around asymmetric traffic. What is wrong is choosing one and then testing it as though it were the other.
The BGP design is the same in both cases. A public ASN on the edge; a private ASN per fabric switch; eBGP everywhere including inside the fabric, with multipath relaxed so failover is the removal of a next-hop from an existing equal-cost set rather than a best-path re-election.
router bgp 4200000021 ! border leaf 1
bgp router-id 10.255.0.21
bgp bestpath as-path multipath-relax
neighbor FABRIC peer-group
neighbor FABRIC remote-as external
neighbor swp1 interface peer-group FABRIC ! frontend leaf 1 (unnumbered)
neighbor swp2 interface peer-group FABRIC ! frontend leaf 2
neighbor 10.255.11.1 remote-as 64512 ! edge firewall, inside link
neighbor 169.254.31.2 remote-as 4200000041 ! gateway node 1
address-family ipv4 unicast
maximum-paths 64
neighbor 10.255.11.1 route-map RM-FROM-EDGE in ! default only
neighbor 169.254.31.2 route-map RM-FROM-GW in ! service prefixes only
address-family l2vpn evpn
neighbor FABRIC activate
advertise-all-vni
!
bfd
peer 10.255.11.1
What each session is allowed to carry is the real design. Route policy here is blast-radius control, not tidiness:
| Session | Advertise | Accept |
|---|---|---|
| Edge ↔ transit provider | Only the locally originated public aggregate | Default route only, with a max-prefix limit |
| Edge → border pair | Default route only, and only while that unit's own carrier path is usable | Public service prefixes originated inside |
| Border ↔ frontend (underlay) | Loopbacks and VTEP addresses | Same |
| Border ↔ frontend (overlay) | EVPN type-2 and type-5 routes | Same |
| Gateway → border | Elastic-IP, NAT-pool and load-balancer prefixes, anycast from both nodes | Default, tenant virtual-network routes |
Reject a default route from anything except the edge. Reject anything but service space from the gateway nodes. Disable graceful restart toward the firewalls specifically — graceful restart exists to preserve forwarding across a control-plane restart, and that is precisely the behaviour you do not want when the device in question has actually failed.
Convergence is a budget, and it should be written down as targets to be measured rather than guarantees. With BFD at 250 ms × 3 on every session, a fibre or optic failure inside the fabric converges in tens of milliseconds because it is carrier loss plus the removal of one member from an equal-cost set. A leaf switch failure converges in under a second. A gateway node failure narrows an anycast set and lands near a second, with stateful sessions on the failed node re-establishing. Carrier-level failures are the slow ones: egress moves in under a second once the default is withdrawn, but ingress is governed by upstream withdrawal and is measured in seconds to minutes, which is why inbound traffic engineering policy is staged and kept warm rather than configured during an incident.