Protocol Atlas

OVERLAY · mpls · reviewed

VPLS

Virtual Private LAN Service

Emulates one Ethernet segment across a provider network by meshing pseudowires between sites and learning MAC addresses from them.

Presenter modeEmbed this figure

Topological Model for VPLSRFC 4762 · PROPOSED STANDARD · January 2007LDP-Based Signaling of DemultiplexersRFC 4762 · PROPOSED STANDARD · January 2007Auto-DiscoveryRFC 4761 · PROPOSED STANDARD · January 2007

Why it exists

An L3VPN gives a customer routing between sites. Some customers want the sites in one broadcast domain instead — for a cluster heartbeat, a protocol that will not route, or simply because the application was built that way.

VPLS emulates a single Ethernet segment. Each provider edge behaves like a bridge whose ports are pseudowires to the other sites, learning MAC addresses from traffic and flooding what it has not learned. Topological Model for VPLSRFC 4762 · PROPOSED STANDARD · January 2007

EVPN has largely replaced it for new deployments, because advertising MAC addresses in BGP removes the need to learn them by flooding — and with them the all-active multihoming problem VPLS never solved.

A switch made of tunnels

Each provider edge behaves like a bridge whose ports are pseudowires. Split horizon on the mesh does the job spanning tree would otherwise have to.

Site AWants: one L2 domainPE1PE2PE3Site Battachment circuitpseudowirepseudowirepseudowireattachment circuit
  • Link
  • Blocking
  • Packet in flight
  • Discarded
  • Emphasis
Select a device to read its state. Arrow keys walk the topology.
Text equivalent of this diagram
Devices and links at this step
ElementKindState
Site AzoneWants: one L2 domain
PE1router—
PE2router—
PE3router—
Site Bzone—
Site A — PE1linkup · attachment circuit
PE1 — PE2linkup · pseudowire
PE1 — PE3linkup · pseudowire
PE2 — PE3linkup · pseudowire
PE2 — Site Blinkup · attachment circuit
1 / 6

The customer wants one broadcast domain across three sites. Not routing between them — one segment, as though the sites were patched into the same switch.

What changed

  • Site A: Wants → one L2 domain
  • Emphasis on Site A

How it works

The pseudowires between participating PEs form a full mesh. A frame arriving on one pseudowire is never forwarded out another — split horizon on the mesh is the entire loop-prevention design, and it works only because every PE already received the frame directly. Loop free VPLSRFC 4762 · PROPOSED STANDARD · January 2007

Learning is in the data plane. A PE records the source MAC of every frame against the circuit or pseudowire it arrived on, floods unknown unicast and broadcast to every pseudowire, and ages entries out. Address LearningRFC 4762 · PROPOSED STANDARD · January 2007

Signalling comes in two forms with one data plane. LDP names each pseudowire explicitly; BGP auto-discovers members and advertises label blocks, so adding a site is a change on that site alone. Auto-DiscoveryRFC 4761 · PROPOSED STANDARD · January 2007

MAC withdrawal lets a PE tell the others to flush specific addresses after a topology change, rather than waiting out the ageing timer with traffic going to the wrong site. MAC Address WithdrawalRFC 4762 · PROPOSED STANDARD · January 2007

On the wire

Constructed examples, encoded from the field table below them — not captured traffic.

Outer label 16001 (S=0) toward the egress PE, inner VPN label 5001 (S=1) selecting the VRF.

MPLS
Stacked after the link-layer header, before the inner IP or Ethernet payload. RFC 3032

Configure it

BGP-signalled VPLS, so adding a site is one change on that site.

Juniper Junos 23.4 · MX204, MX960draft

  1. set routing-instances CUST-A instance-type vpls
set routing-instances CUST-A interface ge-0/0/1.0
set routing-instances CUST-A route-distinguisher 65000:100
set routing-instances CUST-A vrf-target target:65000:100

    The route target is what defines membership. Every PE with the same target discovers the others through BGP — no peer is named anywhere.

    RFC 4761 §3.1

  2. set routing-instances CUST-A protocols vpls site SITE-A site-identifier 1
set routing-instances CUST-A protocols vpls site-range 32

    The site identifier is the VE ID. It must be unique in the instance and reasonably dense: the label block is sized by the range, so a sparse allocation wastes label space at every PE.

    Common mistake: Reusing a site identifier at two PEs. The two sites then compute each other’s labels wrongly and the pseudowire between them never carries traffic correctly.

    RFC 4761 §3.2.1

  3. set forwarding-options storm-control-profiles PROTECT all bandwidth-percentage 1
set routing-instances CUST-A interface ge-0/0/1.0 forwarding-options storm-control-profile PROTECT

    The line that decides whether one customer’s bridging loop becomes twenty sites’ outage. There is no spanning tree in the core to stop it.

    RFC 4762 §4.1

  4. set routing-instances CUST-A protocols vpls mac-table-size 500
set routing-instances CUST-A protocols vpls interface ge-0/0/1.0 mac-limit 100

    Every PE holds every MAC in the instance, so one customer’s large or misbehaving domain fills the table for everyone on that PE. The limit is a service parameter, not a safety net.

    RFC 4762 §4.2

  5. set routing-instances CUST-A protocols vpls mac-flush

    Sends an explicit flush after a topology change instead of waiting out the ageing timer with traffic going to the site the host has left.

    RFC 4762 §6.2

Verify

show vpls connections
Every pseudowire and its state — a partial mesh is visible here and nowhere else.
show vpls mac-table
What this PE has learned, and against which circuit or pseudowire.
show vpls statistics
Flood volume, which is what rises before an incident.
show route table CUST-A.l2vpn.0
The BGP advertisements that discovered the members.

Caveats

  • Split horizon assumes a full mesh. Two PEs that reach a third but not each other simply cannot communicate, and every other check looks healthy.
  • Site identifiers must be unique and dense; the label block is sized from the range.
  • Without storm control at the edge a customer loop is replicated to every site.

When it breaks

Symptom first, because that is what you have when it happens.

  1. Every site of one customer degrades at once, and the provider core shows a large rise in broadcast traffic.

    Narrow it down

    1. Find the attachment circuit with the highest broadcast rate — the storm has one origin.
    2. Check whether MAC tables at remote PEs are churning between pseudowires.
    3. Confirm whether storm control is configured on that circuit.

    Cause

    A bridging loop inside one customer site. VPLS replicates broadcast to every pseudowire because that is what an emulated LAN does, so one site’s loop becomes every site’s loop.

    Fix

    Rate-limit broadcast and unknown unicast per attachment circuit, and set a MAC limit. Containment is at the edge; there is nothing in the core to stop it.

    Flooding and ForwardingRFC 4762 · PROPOSED STANDARD · January 2007

  2. Two sites cannot reach each other while both reach every other site normally.

    Narrow it down

    1. List the pseudowires at each PE and compare against the membership list.
    2. Look for a pseudowire configured at one end only.
    3. Check whether a third PE has both as neighbours, which is what makes it look healthy.

    Cause

    The mesh is not full. Split horizon forbids forwarding between pseudowires, so a PE that can reach both cannot relay between them.

    Fix

    Complete the mesh, or move to BGP auto-discovery so membership is advertised rather than configured.

    Loop free VPLSRFC 4762 · PROPOSED STANDARD · January 2007

  3. A PE stops learning new addresses and traffic for known hosts starts flooding.

    Narrow it down

    1. Check the MAC table size against the platform limit.
    2. Group the entries by attachment circuit to find which customer is responsible.
    3. Look for MAC churn, which inflates the table with the same addresses.

    Cause

    Every PE holds every customer MAC in the instance. One customer with a large or misbehaving layer-two domain fills the table for everyone on that PE.

    Fix

    Set a per-circuit MAC limit, and treat the limit as a service parameter rather than a safety net.

  4. A host moves between sites and traffic keeps going to the old one for minutes.

    Narrow it down

    1. Check whether MAC withdrawal is enabled and being sent.
    2. Compare the ageing timer against the observed delay.
    3. Look for a gratuitous ARP that would have corrected it and did not arrive.

    Cause

    Without MAC withdrawal the old entry has to age out. Data-plane learning cannot know a move happened until something arrives from the new location.

    Fix

    Enable MAC withdrawal so a topology change flushes the affected addresses deliberately.

    MAC Address WithdrawalRFC 4762 · PROPOSED STANDARD · January 2007

  5. A pseudowire is signalled and shown as up, and no traffic crosses it.

    Narrow it down

    1. Check whether a transport LSP exists to the remote provider edge’s loopback.
    2. Confirm the LDP session that signalled the pseudowire is targeted rather than link-local.
    3. Look at whether the pseudowire label is being imposed on a stack with nothing underneath it.

    Cause

    A pseudowire needs two labels: its own, and a transport label to reach the far PE. Signalling establishes the first, and the second comes from the underlying MPLS network — which can be missing entirely while the pseudowire reports itself healthy.

    Fix

    Verify the transport LSP to the far loopback separately from the pseudowire. They fail independently and only one of them says so.

    LDP-Based Signaling of DemultiplexersRFC 4762 · PROPOSED STANDARD · January 2007

Design notes

Put storm control and a MAC limit on every attachment circuit before the service is turned up. A loop inside one customer site is replicated to every other site by design, and the provider core has no spanning tree to save it.

A partial mesh is worse than a missing site. Split horizon assumes a full mesh, so two PEs that peer with a third but not with each other simply cannot reach each other — and every other check looks healthy.

Prefer BGP signalling where BGP is already deployed. The data plane is identical, and the difference is whether adding a site is one change or one change per existing site.

For a new build, look at EVPN first. Learning MACs in the control plane removes the flooding, gives all-active multihoming, and turns MAC mobility into an advertisement rather than a race between ageing timers.

Misconceptions

“Spanning tree protects a VPLS from loops.”
There is no spanning tree in the core. Loop prevention is split horizon on the pseudowire mesh, which handles loops between pseudowires and nothing else. Customer BPDUs are carried as data, not acted upon. Loop free VPLSRFC 4762 · PROPOSED STANDARD · January 2007
“BGP-signalled and LDP-signalled VPLS are different services.”
The data plane is the same Ethernet-over-MPLS pseudowire. What differs is how members find each other and how labels are allocated — a provisioning difference, not a forwarding one. Auto-DiscoveryRFC 4761 · PROPOSED STANDARD · January 2007
“EVPN is VPLS with BGP signalling.”
BGP-signalled VPLS already exists and still learns MACs by flooding. EVPN advertises MAC addresses in BGP, which is what removes the flooding, enables all-active multihoming, and makes mobility an advertisement instead of a race between ageing timers.
“A VPLS instance is just a VLAN stretched over the WAN.”
It behaves like one and is built completely differently: a mesh of pseudowires, per-PE MAC tables learned from tunnelled traffic, and no shared spanning tree. The differences surface exactly when something goes wrong.

More walkthroughs

Two ways to build the same meshdesign-choice

LDP signals each pseudowire and needs every peer named. BGP discovers the peers and hands out label blocks, so adding a site touches one device.

New PELDP: 20 neighbours to configureRoute reflectorTwenty existing PEs20 touches
  • Link
  • Blocking
  • Packet in flight
  • Discarded
  • Emphasis
Select a device to read its state. Arrow keys walk the topology.
Text equivalent of this diagram
Devices and links at this step
ElementKindState
New PErouterLDP: 20 neighbours to configure
Route reflectorrouter—
Twenty existing PEszone—
New PE — Route reflectorlinkup
Route reflector — Twenty existing PEslinkup
New PE — Twenty existing PEslinkstandby
1 / 6

With LDP signalling each pseudowire is configured at both ends. A twenty-first PE means twenty new configurations on twenty existing routers.

What changed

  • New PE: LDP → 20 neighbours to configure
  • Emphasis: 20 touches

The customer looped their switch and took out the WANfailure

A broadcast storm in one site is replicated to every other site, because that is what an emulated LAN is for.

Site A · loopedBridging loopBroadcast rate: risingPE1Provider coreEvery other site
  • Link
  • Blocking
  • Packet in flight
  • Discarded
  • Emphasis
Select a device to read its state. Arrow keys walk the topology.
Text equivalent of this diagram
Devices and links at this step
ElementKindState
Site A · loopedzoneBroadcast rate: rising
PE1router—
Provider corecloud—
Every other sitezone—
Site A · looped — PE1linkup
PE1 — Provider corelinkup
Provider core — Every other sitelinkup
1 / 5

Someone patches two access switches together at Site A. The customer’s own spanning tree should catch it; it has been disabled on that VLAN.

What changed

  • Emphasis: Bridging loop
  • Site A · looped: Broadcast rate → rising

Terms

Pseudowire
An emulated point-to-point circuit over an MPLS tunnel. In VPLS the pseudowires between provider edges are the ports of an emulated switch.
VPLS split horizon
The rule that a frame arriving on one pseudowire is never sent out another. It is the only loop prevention in a VPLS core, and it works only because the mesh is full.
MAC withdrawal
A message telling other provider edges to flush specific addresses after a topology change, instead of waiting out the ageing timer while traffic goes to the site a host has left.

Check yourself

  • What does a provider edge do with a frame whose destination MAC it has not learned?
  • What prevents loops in a VPLS core?
  • What differs between BGP-signalled and LDP-signalled VPLS?
  • Two sites cannot reach each other; both reach every other site. What is wrong?
  • What does EVPN change that BGP-signalled VPLS does not?
  • Where does a VPLS provider edge learn remote MAC addresses?
  • What problem does hierarchical VPLS address?
  • A host moves between sites and traffic goes to the old one for minutes. What is missing?