L3 · igp · reviewed

BFD

Bidirectional Forwarding Detection

Detects that a path has stopped forwarding in milliseconds, so routing protocols can stop inferring it from missed hellos seconds later.

Presenter modeEmbed this figure

Functional SpecificsRFC 5880 · PROPOSED STANDARD · June 2010EncapsulationRFC 5881 · PROPOSED STANDARD · June 2010IssuesRFC 5883 · PROPOSED STANDARD · June 2010BFD on LAG Member LinksRFC 7130 · PROPOSED STANDARD · February 2014

Why it exists

A routing protocol notices a dead neighbour by not hearing from it. OSPF waits four hello intervals, BGP waits for a hold timer — tens of seconds, during which traffic is being sent into a path that is not forwarding.

Where a router can see the link go down it converges immediately. The problem is every case where it cannot: a carrier transport, a switch in the middle, a tunnel, or a fibre that has failed in one direction only. In all of them the interface stays up and the only evidence is silence. IntroductionRFC 5880 · PROPOSED STANDARD · June 2010

The obvious alternative — much faster hellos — does work and scales badly. Hello processing is routing-protocol work, and a router with hundreds of neighbours cannot afford it at sub-second intervals.

Detecting in milliseconds what a hello timer takes seconds to notice

A routing protocol infers failure from silence. BFD makes the silence short enough to act on without making the routing protocol itself chatty.

R1 and R2 are OSPF neighbours across a carrier’s transport. The interfaces terminate on the carrier’s equipment, so a failure in the middle leaves both interfaces up. R1: OSPF hello 10 s, Dead interval 40 s. Carrier transport. R2.

R1OSPF hello: 10 sDead interval: 40 sCarrier transportFailure here is invisible to both endsR2
  • Link
  • Blocking
  • Packet in flight
  • Discarded
  • Emphasis
Select a device to read its state. Arrow keys walk the topology.
Text equivalent of this diagram
Devices and links at this step
ElementKindState
R1routerOSPF hello: 10 s · Dead interval: 40 s
Carrier transportcloud
R2router
R1Carrier transportlinkup
Carrier transportR2linkup
1 / 6

R1 and R2 are OSPF neighbours across a carrier’s transport. The interfaces terminate on the carrier’s equipment, so a failure in the middle leaves both interfaces up.

This is the case BFD was built for. Where a router can see a link go down, it converges immediately; where it cannot, it waits for a timer.

What changed

  • R1: OSPF hello → 10 s
  • R1: Dead interval → 40 s
  • Failure here is invisible to both ends

How it works

Two systems establish a session and send small packets at a negotiated interval. Each advertises what it can transmit and what it is willing to receive, and the session runs at the slower of the two — so one weak neighbour sets the pace rather than being overwhelmed. Transmitting BFD Control PacketsRFC 5880 · PROPOSED STANDARD · June 2010

Detection time is the negotiated interval multiplied by the detect multiplier the far end advertised. Three missed packets at 300 milliseconds is 900 milliseconds, and that number is the entire tuning surface. Calculating the Detection TimeRFC 5880 · PROPOSED STANDARD · June 2010

BFD makes no routing decision. It tells registered clients that a path is down and they react — OSPF drops the adjacency, BGP tears down the session. One BFD session can serve every protocol on that path, so the cost is paid once. IntroductionRFC 5880 · PROPOSED STANDARD · June 2010

On the wire

Constructed examples, encoded from the field table below them — not captured traffic.

Both discriminators are populated, so each end knows the other. Desired TX 100 000 microseconds with a multiplier of three is a 300 ms detection time — the whole tuning surface in two fields.

IPv4 or IPv6
Single-hop sessions use UDP destination port 3784 with TTL 255, so a packet that has been routed cannot be mistaken for one from a directly connected neighbour. RFC 5880
Mandatory section
Version, diagnostic, state, flags, detect multiplier, length, discriminators and three intervals. RFC 5880
Authentication section
Optional and variable, with a format that depends on the authentication type in use. RFC 5880

Configure it

Detect a transport failure in under a second and tell both routing protocols.

Cisco IOS-XE 17.12 · Catalyst 8500, ASR 1000draft

  1. interface GigabitEthernet0/0/1
     bfd interval 300 min_rx 300 multiplier 3

    Transmit interval, minimum receive interval, and how many may be missed. Detection time is 900 ms — the product, which is the number worth stating in a design document.

    Common mistake: Reading `interval` as the detection time. It is one packet spacing; the multiplier is what turns it into a detection time.

    RFC 5880 §6.8.4

  2. router ospf 1
     bfd all-interfaces

    Registers OSPF as a client. Without this the session comes up, reports correctly, and nothing reacts when it fails.

    Common mistake: Configuring the interface and stopping. `show bfd neighbors` looks healthy and convergence is unchanged, which makes it look like BFD does not work.

  3. router bgp 65001
     neighbor 10.255.0.1 fall-over bfd

    Registers BGP against the same session. One session, two clients — the cost is paid once.

  4. bfd-template single-hop DAMPEN
     interval min-tx 300 min-rx 300 multiplier 3
     dampening half-life 30 reuse 1000 suppress 2000 max-suppress-time 120

    Holds a repeatedly flapping session down. Essential wherever the timers are aggressive, because a marginal link otherwise costs a reconvergence per flap.

Verify

show bfd neighbors detail
Negotiated intervals, detection time, and the registered clients.
show bfd summary
Session counts and states.

Caveats

  • The negotiated interval can be slower than the configured one; the neighbour’s minimum receive interval sets the floor.
  • Whether sessions run in hardware, on the line card, or in software is platform-specific and decides how aggressive the timers can safely be.

When it breaks

Symptom first, because that is what you have when it happens.

  1. A session establishes and runs far slower than configured, and detection is nowhere near the expected time.

    Narrow it down

    1. Read the negotiated interval on both ends, not the configured one.
    2. Compare each end’s minimum receive interval.
    3. Check whether one platform silently clamps to a floor.

    Cause

    The interval is negotiated to the slower of the two. A neighbour advertising a minimum receive interval of one second caps the session there regardless of what the other end asked for.

    Fix

    Configure both ends consistently, and verify the negotiated value rather than the intended one. Many platforms enforce a minimum well above what the configuration accepts.

    Transmitting BFD Control PacketsRFC 5880 · PROPOSED STANDARD · June 2010
  2. A degraded link produces continuous reconvergence, and the network is worse than it would be with the link simply left up.

    Narrow it down

    1. Correlate session state changes against interface error counters.
    2. Check the detection time against the observed loss rate.
    3. Count SPF runs or BGP updates during the event.

    Cause

    Detection is fast enough that ordinary packet loss looks like failure. Each false detection costs a full reconvergence, and the link recovers immediately and does it again.

    Fix

    Lengthen the interval to survive normal loss, and enable dampening so a repeatedly flapping session is held down.

  3. Enabling BFD widely raises control-plane CPU, and sessions start flapping under load in a way that looks like a network problem.

    Narrow it down

    1. Establish whether sessions are processed in hardware, on the line card, or in software.
    2. Count sessions and multiply by the transmit rate.
    3. Check whether flaps correlate with CPU rather than with interface errors.

    Cause

    Software-processed sessions at aggressive intervals. The CPU cannot keep up, packets are late, and the sessions declare each other down — a failure caused entirely by the detection mechanism.

    Fix

    Reduce the number of sessions or lengthen the interval, and prefer platforms that offload. Where offload exists it is usually restricted to single-hop sessions on physical interfaces.

  4. A BFD session is Up and detects a failure in milliseconds, and the routing protocol still takes seconds to react.

    Narrow it down

    1. Check which protocols are registered as clients of the session, not just that the session exists.
    2. Confirm the routing protocol was told to use BFD on that interface or for that neighbour.
    3. Look at whether the session came up before or after the protocol adjacency.

    Cause

    BFD detects; it does not act. A session with no client protocol registered runs perfectly, logs its state changes, and tells nobody who could do anything about it.

    Fix

    Register the client explicitly — under the routing protocol, per interface or per neighbour depending on the platform. Configuring BFD on an interface is not the same as asking a protocol to use it.

    Control Protocol InteractionsRFC 5882 · PROPOSED STANDARD · June 2010
  5. A bundle stays up and a proportion of flows are black-holed, roughly matching one member’s share.

    Narrow it down

    1. Check whether BFD runs across the bundle or per member.
    2. Compare per-member counters for one that is receiving nothing.
    3. Test flows that hash to different members.

    Cause

    A single session over the bundle. It succeeds as long as any member forwards, so a failed member is invisible while the flows hashed to it disappear.

    Fix

    Use micro-BFD, which runs a session per member and removes only the failed one.

    BFD on LAG Member LinksRFC 7130 · PROPOSED STANDARD · February 2014

Design notes

Match the interval to the failure mode. A carrier transport that fails cleanly justifies aggressive timers; a link that degrades before it dies needs enough tolerance to ride out ordinary loss, because declaring it dead removes a path that was still carrying most traffic.

Enable dampening wherever timers are aggressive. Without it a marginal link oscillates, and each cycle costs a full reconvergence across the area — which is more expensive than the loss that triggered it.

Check where BFD is processed before committing to sub-second timers. On some platforms it runs in hardware or on the line card and the intervals are honest; on others it runs in software on the control plane, and a hundred sessions at 50 milliseconds is a self-inflicted denial of service.

On a link aggregation group, use micro-BFD. A single session across the bundle cannot tell which member failed, and the bundle stays up while one member black-holes the flows hashed to it. BFD on LAG Member LinksRFC 7130 · PROPOSED STANDARD · February 2014

Misconceptions

BFD is a routing protocol.
It carries no routing information and makes no decisions. It answers one question — is this path forwarding — and tells whichever protocols asked. Those protocols do the reacting. IntroductionRFC 5880 · PROPOSED STANDARD · June 2010
Lower intervals are always better.
Below the loss characteristics of the link, faster detection converts degradation into oscillation. A path losing two percent of packets carries most traffic; a path being declared down every second carries none.
BFD replaces the routing protocol’s hellos.
It runs alongside them. The protocol keeps its own timers as a backstop and simply acts on BFD first, which is why the hello interval usually stays at its default when BFD is deployed.

More walkthroughs

Both ends agreeing on how fast to faildesign-choice

A session comes up through three states, and the interval it settles on is not the one either end configured.

Both ends start in Down and send packets saying so. Neither has heard from the other, so Your Discriminator is zero in both directions. Fast router: State Down. Slower router: State Down.

Fast routerState: DownSlower routerState: Down
  • Link
  • Blocking
  • Packet in flight
  • Discarded
  • Emphasis
Select a device to read its state. Arrow keys walk the topology.
Text equivalent of this diagram
Devices and links at this step
ElementKindState
Fast routerrouterState: Down
Slower routerrouterState: Down
Fast routerSlower routerlinkup
1 / 6

Both ends start in Down and send packets saying so. Neither has heard from the other, so Your Discriminator is zero in both directions.

A session that never leaves Down, with packets arriving and Your Discriminator staying zero, means one direction is not getting through. That single field distinguishes a one-way path from a session nobody configured.

What changed

  • Fast router: State → Down
  • Slower router: State → Down
  • Down, yours = 0: Fast router → Slower router

Detection so fast it makes things worsefailure

A marginal link plus aggressive timers turns a degraded path into continuous reconvergence, which costs more than the packet loss did.

BFD is configured at 50 milliseconds with a multiplier of 3. Detection time is 150 milliseconds, which is genuinely fast. R1: BFD interval 50 ms, Detection time 150 ms. R2. R3.

R1BFD interval: 50 msDetection time: 150 msR2R3marginal
  • Link
  • Blocking
  • Packet in flight
  • Discarded
  • Emphasis
Select a device to read its state. Arrow keys walk the topology.
Text equivalent of this diagram
Devices and links at this step
ElementKindState
R1routerBFD interval: 50 ms · Detection time: 150 ms
R2router
R3router
R1R2linkup · marginal
R1R3linkup
R3R2linkup
1 / 5

BFD is configured at 50 milliseconds with a multiplier of 3. Detection time is 150 milliseconds, which is genuinely fast.

What changed

  • R1: BFD interval → 50 ms
  • R1: Detection time → 150 ms
  • Emphasis on R1 ↔ R2

Terms

Detection time
The negotiated BFD interval multiplied by the detect multiplier. The whole tuning surface: it is what should be compared against the failure being protected against.
BFD echo
A mode where a system sends packets the neighbour loops straight back in the forwarding plane, testing the data path rather than the neighbour’s control plane.

Check yourself

  • What problem does BFD solve that a link-down signal does not?
  • A session negotiates 300 ms with a multiplier of 3. What is the detection time?
  • What does BFD do when it detects a failure?
  • Two ends configure different intervals. What does the session run at?
  • A link losing 2% of packets causes continuous reconvergence with 150 ms detection. What is the fix?
  • Packets are arriving from the neighbour and Your Discriminator is zero in all of them. What does that mean?
  • One end asks for 50 ms and the other can only receive at 300 ms. What interval is used?
  • A session went down with diagnostic 3, "neighbor signaled session down". What does that rule out?