OVERLAY · overlay · reviewed
EVPN
Ethernet VPN
Advertises MAC addresses in BGP instead of discovering them by flooding, which is what makes all-active multihoming and predictable mobility possible.
Presenter modeEmbed this figure
Why it exists
Every earlier way of stretching a layer-two segment learned MAC addresses by flooding. That works, and it means a host nobody has heard from is unreachable, a topology change is resolved by ageing timers, and two switches attached to one host cannot both forward.
EVPN moves that information into BGP. A leaf that sees a host advertises it; every other leaf installs it from a routing update. The data plane is unchanged — usually VXLAN, sometimes MPLS — and what changes is where knowledge comes from. MAC/IP Advertisement RouteRFC 7432 · PROPOSED STANDARD · February 2015
The consequences are larger than the mechanism suggests. Advertising a MAC makes all-active multihoming expressible, makes mobility an ordered event rather than a race, and turns the flood list into something the control plane maintains.
A MAC address as a routing update
The leaf that sees a host tells everyone. Remote leaves install the MAC from a BGP route rather than learning it from a flooded frame.
Host A sends its first frame — usually a gratuitous ARP. Leaf 1 learns the MAC on the access port exactly as any switch would. Host A. Leaf 1: Local MAC A on Eth1/1. Route reflector. Leaf 2. Host B.
- Link
- Blocking
- Packet in flight
- Discarded
- Emphasis
Text equivalent of this diagram
| Element | Kind | State |
|---|---|---|
| Host A | host | — |
| Leaf 1 | switch | Local: MAC A on Eth1/1 |
| Route reflector | router | — |
| Leaf 2 | switch | — |
| Host B | host | — |
| Host A — Leaf 1 | link | up |
| Leaf 1 — Route reflector | link | up |
| Route reflector — Leaf 2 | link | up |
| Leaf 2 — Host B | link | up |
Host A sends its first frame — usually a gratuitous ARP. Leaf 1 learns the MAC on the access port exactly as any switch would.
What changed
- Leaf 1: Local → MAC A on Eth1/1
- Emphasis on Host A ↔ Leaf 1
How it works
A type 2 route carries a MAC, optionally the IP it was seen with, and the VNI. The IP is what lets a remote leaf answer ARP locally instead of flooding it, which removes most fabric-wide broadcast. Remote LearningRFC 7432 · PROPOSED STANDARD · February 2015
A type 3 route says a leaf wants broadcast traffic for a VNI. Collected, those routes are the ingress replication list — the same list a flood-and-learn fabric maintains by hand and forgets to update. Constructing Inclusive Multicast Ethernet Tag RouteRFC 7432 · PROPOSED STANDARD · February 2015
Multihoming keys entirely on the Ethernet segment identifier. Type 4 routes discover peers on the segment and elect a designated forwarder; the ESI label lets a peer drop a frame the host originated; type 1 routes give remote leaves both paths and let a link failure be withdrawn once rather than once per MAC. Aliasing and Backup PathRFC 7432 · PROPOSED STANDARD · February 2015
Routing between segments is IRB. Asymmetric routes at ingress and bridges at egress, so every leaf needs every VNI it might route into; symmetric routes at both ends through a transit L3VNI, so a leaf needs only the VNIs it has hosts in. Symmetric and Asymmetric IRBRFC 9135 · PROPOSED STANDARD · October 2021
On the wire
Constructed examples, encoded from the field table below them — not captured traffic.
- BGP path attribute
- Optional non-transitive. Type code 14. RFC 4760
Configure it
evpn vni 10010 l2 rd auto route-target import auto route-target export auto`auto` derives both the distinguisher and the targets from the AS and the VNI, so every leaf computes the same value. Configuring one leaf by hand and the rest automatically is the quiet EVPN failure: sessions up, routes advertised, nothing imported.
RFC 8365 §5.1.2.1
vrf context TENANT-A vni 50001 rd auto address-family ipv4 unicast route-target both auto evpn interface nve1 member vni 50001 associate-vrfThe transit L3VNI symmetric IRB routes through. It has to exist on every leaf that participates — a leaf without it bridges perfectly and cannot route, which is the exact symptom of a fabric that "works within a VNI".
fabric forwarding anycast-gateway-mac 0000.2222.3333 interface Vlan10 vrf member TENANT-A ip address 10.10.10.1/24 fabric forwarding mode anycast-gatewayThe same gateway address and MAC on every leaf, so a host keeps its default gateway when it moves. Without the shared MAC a move invalidates the host’s ARP cache and traffic stops until it re-resolves.
RFC 7432 §10.1
interface port-channel10 switchport mode trunk ethernet-segment 10 esi 0300.0000.0000.0000.0001The ESI names the segment. This exact value must appear on the peer leaf too — byte for byte. A mismatch means no designated forwarder election and no split horizon, with nothing logged on either side.
Common mistake: Deriving the ESI from the switch — its serial, its address, its number. It identifies the link bundle, so both leaves must carry the same one.
RFC 7432 §5
evpn vni 10010 l2 mac duplication limit 5 time 180 action freezeThe backstop for an address that keeps moving. Freezing protects the control plane and leaves the address unreachable, so alert on it — a frozen MAC is always a symptom of something else.
RFC 7432 §15.1
router bgp 65000 neighbor 10.0.0.100 remote-as 65000 address-family l2vpn evpn send-community extended`send-community extended` is what carries the mobility sequence and the ESI-derived targets. Without it, mobility silently stops being ordered and every move looks like a tie.
RFC 7432 §15
Verify
show bgp l2vpn evpn summary- That the address family is negotiated, not just the session.
show nve ethernet-segment- The ESI, the peer leaf, and which VLANs this leaf is DF for.
show l2route evpn mac all- Remote MACs and the sequence number each carries.
show bgp l2vpn evpn <mac>- The extended communities, which is where a stripped one shows up.
Caveats
- Mixing auto-derived and hand-configured route targets produces a fabric that advertises correctly and imports nothing.
- The ESI must be identical on both leaves of a multihomed pair; a mismatch is silent on both.
- Without extended communities the mobility sequence is lost and moves stop converging.
When it breaks
Symptom first, because that is what you have when it happens.
A host moves between leaves and traffic keeps going to the old one for several minutes.
Narrow it down
- Check the MAC mobility sequence number on the type 2 route at both leaves.
- Confirm the new leaf advertised with a higher sequence than the old one.
- Look for the old leaf still advertising and not withdrawing.
Cause
Mobility is resolved by a sequence number carried in an extended community. If the new leaf does not increment it, or a device in the path strips the community, remote leaves have nothing to compare and keep what they had.
Fix
Verify the mobility extended community survives end to end. A route reflector that drops unknown extended communities produces exactly this.
MAC MobilityRFC 7432 · PROPOSED STANDARD · February 2015Duplicate packets arrive at a multihomed host, or some flows toward it disappear entirely.
Narrow it down
- Compare the Ethernet segment identifier on each leaf of the pair — they must match exactly.
- Check which leaf won designated forwarder election for each VLAN.
- Verify type 4 routes are being received by both leaves.
Cause
The two leaves do not agree they are attached to the same segment. With mismatched ESIs each considers itself the only attachment, both forward broadcast, and split-horizon filtering never engages.
Fix
Make the ESI identical on both leaves. It identifies the segment, not the leaf, and that is the mistake almost every time.
Designated Forwarder ElectionRFC 7432 · PROPOSED STANDARD · February 2015Hosts in the same VNI reach each other but cannot reach another VNI, despite a gateway being configured.
Narrow it down
- Establish whether the fabric uses symmetric or asymmetric IRB.
- For symmetric, check the L3VNI is configured and consistent on every leaf involved.
- Check whether type 5 routes are present for the remote prefix.
Cause
Symmetric IRB routes between VNIs through a transit L3VNI that has to exist at both ends. A leaf missing it can bridge and cannot route, which is exactly this asymmetry.
Fix
Configure the L3VNI and its VRF consistently everywhere, and confirm type 5 routes are advertised and imported.
Symmetric IRB ProceduresRFC 9135 · PROPOSED STANDARD · October 2021ARP resolution is slow or intermittent across the fabric, though unicast between resolved hosts is fine.
Narrow it down
- Check whether ARP suppression is enabled and whether the leaf holds the binding.
- Confirm type 2 routes carry the IP address as well as the MAC.
- Look at how broadcast is being replicated for that VNI.
Cause
Suppression answers from EVPN-learned bindings. If type 2 routes carry only the MAC the leaf cannot answer and falls back to flooding — and if replication is also incomplete, resolution depends on which path happens to work.
Fix
Ensure type 2 routes advertise the IP alongside the MAC, and verify the replication list separately.
MAC/IP Advertisement RouteRFC 7432 · PROPOSED STANDARD · February 2015BGP sessions are established, routes are being advertised, and no leaf installs anything.
Narrow it down
- Compare the route targets attached to advertised routes against those imported at the receiver.
- Check whether one side auto-derives the target and the other configures it by hand.
- Look at the received-routes count before policy versus after.
Cause
The import policy does not match what is being exported. Auto-derivation builds the target from the AS and the VNI, so a manual value that differs by one digit is accepted everywhere and imports nothing.
Fix
Auto-derive on every leaf, or configure by hand on every leaf. The failure comes from mixing the two.
Auto-Derivation of RTRFC 8365 · PROPOSED STANDARD · March 2018
Design notes
The ESI names the segment, not the switch. It must be byte-identical on every leaf attached to the same host, and a mismatch produces duplicate delivery and no split horizon with nothing logged anywhere — each leaf’s configuration is internally complete.
Choose symmetric IRB unless there is a specific reason not to. Asymmetric is simpler to explain and needs every routed VNI configured on every leaf, which is the first thing to exhaust leaf table space in a fabric of any size. Symmetric and Asymmetric IRBRFC 9135 · PROPOSED STANDARD · October 2021
Let route targets auto-derive where the platform supports it. Hand-configured targets are the quiet failure of EVPN deployments: everything is configured, the session is established, and nothing imports. Auto-Derivation of RTRFC 8365 · PROPOSED STANDARD · March 2018
Enable duplicate MAC detection and alert on it. It exists to protect the control plane from an address that keeps moving, and a frozen MAC is always a symptom of something else that needs finding. MAC Duplication IssueRFC 7432 · PROPOSED STANDARD · February 2015
Misconceptions
- “EVPN is a data plane.”
- It is a control plane — an address family in BGP. The data plane is usually VXLAN and can be MPLS. What EVPN replaces is flood-and-learn: MAC addresses arrive in routing updates rather than being discovered by flooding. BGP MPLS-Based EVPN OverviewRFC 7432 · PROPOSED STANDARD · February 2015
- “Symmetric and asymmetric IRB are two names for the same thing.”
- Asymmetric routes at ingress and bridges at egress, so every leaf needs every VNI it might route into. Symmetric routes at both ends through a transit L3VNI, so a leaf needs only the VNIs it has hosts in. The second scales; the first is simpler and runs out of table space first. Symmetric and Asymmetric IRBRFC 9135 · PROPOSED STANDARD · October 2021
- “The Ethernet segment identifier identifies a leaf.”
- It identifies the segment — the link bundle a multihomed host is attached by — and must be identical on every leaf attached to it. Treating it as per-device is the most common EVPN multihoming misconfiguration. Ethernet SegmentRFC 7432 · PROPOSED STANDARD · February 2015
- “EVPN requires MPLS, as its RFC title says.”
- RFC 7432 defines it over MPLS because that came first. RFC 8365 defines the same control plane over VXLAN, which is what almost every data centre runs. The encapsulation is a separate choice from the control plane. VXLAN/NVGRE EncapsulationRFC 8365 · PROPOSED STANDARD · March 2018
More walkthroughs
Two leaves, one host, both forwardingdesign-choice
All-active multihoming needs three separate mechanisms to agree, and all three key on one identifier that has to be configured the same on both leaves.
Both leaves are configured with the same Ethernet segment identifier. It names the link bundle, not the switch — and that is the sentence to remember. Multihomed host. Leaf 1: ESI 00:11:...:01. Leaf 2: ESI 00:11:...:01. Leaf 3 · remote.
- Link
- Blocking
- Packet in flight
- Discarded
- Emphasis
Text equivalent of this diagram
| Element | Kind | State |
|---|---|---|
| Multihomed host | switch | — |
| Leaf 1 | switch | ESI: 00:11:...:01 |
| Leaf 2 | switch | ESI: 00:11:...:01 |
| Leaf 3 · remote | switch | — |
| Multihomed host — Leaf 1 | link | up · ESI 00:11...:01 |
| Multihomed host — Leaf 2 | link | up · ESI 00:11...:01 |
| Leaf 1 — Leaf 3 · remote | link | up |
| Leaf 2 — Leaf 3 · remote | link | up |
Both leaves are configured with the same Ethernet segment identifier. It names the link bundle, not the switch — and that is the sentence to remember.
What changed
- Leaf 1: ESI → 00:11:...:01
- Leaf 2: ESI → 00:11:...:01
- Emphasis on Multihomed host ↔ Leaf 1
- Emphasis on Multihomed host ↔ Leaf 2
The address that kept movingfailure
Mobility is decided by a sequence number rather than a clock. When two hosts genuinely share an address, that number climbs until the fabric stops believing it.
A virtual machine migrates from Leaf 1 to Leaf 2. Leaf 2 sees it, and advertises the same MAC with the mobility sequence incremented. Leaf 1. Leaf 2: Type 2 MAC X · seq 1. Route reflector. Leaf 3 · remote.
- Link
- Blocking
- Packet in flight
- Discarded
- Emphasis
Text equivalent of this diagram
| Element | Kind | State |
|---|---|---|
| Leaf 1 | switch | — |
| Leaf 2 | switch | Type 2: MAC X · seq 1 |
| Route reflector | router | — |
| Leaf 3 · remote | switch | — |
| Leaf 1 — Route reflector | link | up |
| Leaf 2 — Route reflector | link | up |
| Route reflector — Leaf 3 · remote | link | up |
A virtual machine migrates from Leaf 1 to Leaf 2. Leaf 2 sees it, and advertises the same MAC with the mobility sequence incremented.
A sequence number rather than a timestamp, because comparing timestamps would need the fabric’s clocks to agree closely enough to order two events milliseconds apart.
What changed
- Leaf 2: Type 2 → MAC X · seq 1
- Emphasis on Leaf 2 ↔ Route reflector
Terms
- ESI
- Ethernet Segment Identifier. EVPN type 1 and type 4 use it for multihoming, aliasing, mass-withdraw, and DF election. It is not a VNI.
- IMET route (type 3)
- An EVPN advertisement saying this leaf wants broadcast and multicast traffic for a VNI. Collected across the fabric, the type 3 routes are the ingress replication list.
- MAC mobility sequence
- A counter in an extended community, incremented each time a MAC moves, that orders two advertisements without needing synchronised clocks. Strip the community anywhere in the path and every move looks like a tie.
- Symmetric IRB
- Routing between segments at both ends through a transit L3VNI, so a leaf needs only the VNIs it has hosts in. Asymmetric routes at ingress and bridges at egress, which needs every routed VNI on every leaf.