MSDP

Why it exists

A rendezvous point is scoped to a domain. That is deliberate — two organisations will not share one, and neither will two administrative regions of a large network. But it means a receiver’s RP has no way to learn about a source that registered somewhere else.

MSDP is a TCP session between rendezvous points that carries one fact: this source is active for this group. Once a receiver’s domain knows the source address, its own routers can build a tree straight to it. RFC 3618 §2OverviewRFC 3618 · EXPERIMENTAL · October 2003

It also solved a second problem it was not designed for. Anycast RP — several routers sharing one RP address — needs those routers to share source state, and an MSDP mesh group between them is how that was done for years.

Telling another domain that a source exists

A source registers with its own RP. That RP announces it to a peer RP, whose receivers then join the source directly — the data never touches either RP.

How it works

An RP that learns of an active source sends a Source-Active message to its peers, naming the source, the group and itself as the originator. Peers cache it, so a receiver joining later does not have to wait for the next periodic announcement. RFC 3618 §4CachingRFC 3618 · EXPERIMENTAL · October 2003

Loop prevention is peer-RPF: an SA is accepted only from the peer that lies on the unicast path back toward the RP that originated it. There is no other check, which is why getting the peering topology wrong produces silence rather than a loop. RFC 3618 §10.1.3Peer-RPF Forwarding RulesRFC 3618 · EXPERIMENTAL · October 2003

A mesh group suspends that check between its members and stops SAs being forwarded between them, on the promise that the members are fully meshed and so cannot form a loop. That is what makes Anycast RP work, and it is a promise the configuration cannot verify. RFC 3618 §10.2MSDP mesh-group semanticsRFC 3618 · EXPERIMENTAL · October 2003

The data itself does not travel over MSDP. The one exception is that an SA may encapsulate the first packets of a stream, so a receiver that joins at the wrong moment does not lose the beginning. RFC 3618 §8Encapsulated Data PacketsRFC 3618 · EXPERIMENTAL · October 2003

On the wire

Constructed examples, encoded from the field table below them — not captured traffic.

Configure it

When it breaks

Symptom first, because that is what you have when it happens.

1. The MSDP session is established, the SA received counter rises, and the SA cache stays empty.

   Narrow it down

   1. Compare the SA received count against the cache size — that pair is the whole diagnosis.
   2. Check the unicast path toward the RP that originates the SAs.
   3. Confirm whether the receiving peer is the next hop along that path.

   Cause

   Peer-RPF discarded them. The SAs arrived from a peer that is not on the path back toward the originating RP, and discarding them is correct behaviour, so nothing is logged.

   Fix

   Peer with the router that is on the path toward the originating RP, or put the peers in a mesh group — accepting that a mesh group suspends loop protection.

   RFC 3618 §10.1.3Peer-RPF Forwarding RulesRFC 3618 · EXPERIMENTAL · October 2003

2. An Anycast RP pair’s MSDP session never establishes, with no useful error.

   Narrow it down

   1. Check the source address configured for the MSDP session on each router.
   2. Confirm it is a unique loopback and not the shared anycast address.
   3. Look at whether the TCP connection is being made to the router itself.

   Cause

   The session was sourced from the anycast address, which both routers own. Each is effectively trying to peer with itself.

   Fix

   Give each RP a second, unique loopback and source MSDP from it. Keep the anycast address for PIM only.

3. Inter-domain multicast works but takes up to a minute to start for every new receiver.

   Narrow it down

   1. Check whether SA caching is enabled on the receiving RP.
   2. Look at the SA advertisement interval on the originating side.
   3. Confirm the delay matches the advertisement period rather than a routing convergence.

   Cause

   Without a cache the receiving RP has to wait for the next periodic SA before it can tell a last-hop router which source to join.

   Fix

   Enable SA caching. The memory cost is bounded by an SA filter, which should be there anyway.

   RFC 3618 §4CachingRFC 3618 · EXPERIMENTAL · October 2003

4. The SA cache holds tens of thousands of entries and the RP’s control plane is under load.

   Narrow it down

   1. Count the cache entries and group them by originating RP.
   2. Check whether an inbound SA filter exists on each peering.
   3. Look for a mesh group whose members are not actually fully meshed.

   Cause

   Either no SA filter, so every source the far side knows is imported, or a mesh group with a broken promise, so SAs circulate among members that are not fully meshed.

   Fix

   Filter inbound and outbound on every peering, and verify that any mesh group really is a full mesh.

   RFC 3618 §7SA Filtering and PolicyRFC 3618 · EXPERIMENTAL · October 2003

5. An MSDP peering drops and re-establishes during periods of heavy multicast, and comes back cleanly each time.

   Narrow it down

   1. Check whether the hold timer is expiring rather than the TCP connection being reset.
   2. Look at the size of the SA messages being exchanged — the maximum is 9 192 octets.
   3. Check whether keepalives are being starved behind a large SA burst on the same connection.

   Cause

   MSDP runs over one TCP connection carrying both SA messages and keepalives. A burst of large SAs can delay a keepalive past the hold timer, and the session drops for a reason that is congestion rather than failure.

   Fix

   Filter the SA volume down so the bursts are smaller. Raising the hold timer treats the symptom and leaves a peering that takes longer to notice a genuine failure.

   RFC 3618 §5.4Peer Hold TimerRFC 3618 · EXPERIMENTAL · October 2003

Design notes

Filter in both directions on every peering. An unfiltered MSDP session imports every active source the far side knows of, and exports every source you know of. The SA cache is where the consequences show up, and by then it is large. RFC 3618 §7SA Filtering and PolicyRFC 3618 · EXPERIMENTAL · October 2003

Source the MSDP session from a unique loopback, never from the anycast RP address. Peering from the shared address is the most common Anycast RP mistake and it presents as a session that will not come up for no visible reason.

If Anycast RP is the only reason MSDP is being deployed, look at RFC 4610 first. It has the RP forward registers to its anycast peers inside PIM, achieving the same thing without a second protocol to configure, filter and monitor. RFC 4610 §3MechanismRFC 4610 · PROPOSED STANDARD · August 2006

If the traffic is one-to-many, consider whether MSDP is needed at all. SSM has no RP, so it has no inter-domain RP problem — the receiver names the source and the join crosses the boundary by itself.

Misconceptions

“Multicast traffic flows between domains over MSDP.”

MSDP carries announcements. Once the receiving domain knows the source address, its own routers build a tree straight to the source and the data follows the shortest path — past both RPs, not through them. Only the encapsulated first packets in an SA are an exception. RFC 3618 §8Encapsulated Data PacketsRFC 3618 · EXPERIMENTAL · October 2003

“If SAs are being discarded the session must be down.”

The session is up and healthy; peer-RPF is a per-message check applied after it. That is why the diagnosis is a rising receive counter beside an empty cache rather than anything in a log. RFC 3618 §10.1.3Peer-RPF Forwarding RulesRFC 3618 · EXPERIMENTAL · October 2003

“Anycast RP requires MSDP.”

It requires the RPs to share source state, and MSDP was the original way to do it. RFC 4610 does the same inside PIM by forwarding registers between anycast peers, which removes a whole protocol from the design. RFC 4610 §3MechanismRFC 4610 · PROPOSED STANDARD · August 2006

“SSM needs MSDP to work across domains.”

SSM has no RP, so there is no inter-RP problem to solve. The receiver already knows the source and the (S,G) join crosses the boundary on its own. MSDP configured for an SSM range does nothing.

More walkthroughs

Terms

Source-Active message

The MSDP announcement that a source is active for a group. It names the source, the group and the RP that originated it — the last of those so receiving peers can run the peer-RPF check.

Peer-RPF

MSDP’s only loop protection: an SA is accepted solely from the peer on the unicast path back toward the RP that originated it. A failure is silent — the session stays up and the cache stays empty.

MSDP mesh group

A set of peers that do not forward SAs to each other and skip the peer-RPF check between themselves. It trades loop protection for a promise of a full mesh, and it is what makes Anycast RP work.

Check yourself

• What does an MSDP Source-Active message carry?

  • The multicast data itself
  • The source, the group, and the RP that originated the announcement
  • A list of receivers
  • The RP-set for the domain
• The MSDP session is established, SAs are arriving, and the cache is empty. Why?

  • The session is flapping
  • Peer-RPF is discarding them — they arrive from a peer that is not on the path toward the originating RP
  • SA caching is disabled
  • The groups are outside the configured range
• What does putting MSDP peers in a mesh group do?

  • Speeds up SA propagation
  • Suspends the peer-RPF check between members and stops SAs being forwarded between them
  • Encrypts the session
  • Shares the RP address between them
• Inter-domain multicast works but every new receiver waits up to a minute. What is missing?

  • A mesh group
  • SA caching on the receiving RP, so it waits for the next periodic announcement
  • An SA filter
  • IGMPv3 on the receiver
• How much MSDP does an SSM deployment need?

  • One session per domain pair
  • None — SSM has no RP, so there is no inter-RP discovery problem
  • A mesh group between all domains
  • Only for IPv6
• Why does one Source-Active message carry an entry count rather than one source?

  • To compress the message
  • So several sources in one domain can share a single RP address field
  • To batch acknowledgements
  • Because TCP requires a minimum size
• An SA message carries an encapsulated data packet. What is it for?

  • It is how all inter-domain multicast traffic travels
  • So a receiver that joins at the wrong moment does not lose the first packets of a stream
  • To verify the source is really sending
  • To measure the path
• Why does MSDP define no acknowledgement of its own?

  • Because SA messages are idempotent
  • Because it runs over TCP, which provides the reliability
  • Because peers are always directly connected
  • Because losing an SA is harmless

Related

• PIM-SM

  Builds multicast trees only where receivers ask for them, meeting sources and receivers at a rendezvous point and then getting out of the way.

• RP discovery

  Every router in a sparse-mode domain has to agree which rendezvous point serves which group. Four mechanisms answer that, and disagreement is silent.

• SSM

  The receiver names the source, so there is no rendezvous point, no shared tree and no source discovery problem to solve.