Unicast is a conversation: one sender, one receiver, and every router knows where the receiver is because its address says so. Multicast is a broadcast that only reaches the people who asked for it. The sender doesn’t know who’s listening, the address doesn’t say where anyone is, and yet every copy has to arrive exactly once. Everything in this article, every protocol and every acronym, exists to solve one of two problems that follow from that: who wants this traffic, and how does it get to them without loops or duplicates.
The first problem belongs to IGMP, between hosts and their first router. The second belongs to PIM, between routers. Everything else (rendezvous points, SSM, BIDIR, snooping) is a refinement of one or the other. I’ll build it up in that order and prove each piece in a lab: IOS-XR at the edge, NX-OS in the fabric and Linux hosts, with packet captures for everything on the wire. It’s the same lab that A trading day in multicast puts under load.
In short
- A group address names a conversation, not a place. 224.0.0.0/4 is multicast; 232.0.0.0/8 is for SSM, 239.0.0.0/8 for private use, 224.0.0.0/24 never leaves the link. On Ethernet, the low 23 bits of the group map to a MAC address, so 32 groups share each MAC.
- IGMP tells the first router who wants what. IGMPv2 says “group G”; IGMPv3 can say “group G, but only from source S”. A querier keeps asking; hosts answer; silence means nobody’s left.
- PIM builds trees between routers using the unicast routing table backwards. Reverse path forwarding (RPF): accept a packet only on the interface you’d use to reach its source.
- ASM needs a rendezvous point; SSM doesn’t. In ASM, receivers meet sources at the RP and then usually switch to the source’s shortest path. In SSM, the receiver names the source and the tree goes straight to it.
- BIDIR is one tree for many senders. No per-source state; a designated forwarder per link carries traffic towards the RP.
- RPs can be made redundant with Anycast RP (MSDP or PIM), and discovered statically, with Auto-RP or with BSR.
The lab
Two IOS-XR 7.11 routers (XR-A, XR-B) connect a source host, EXCH, to an NX-OS 10.4 fabric of two spines and two leaves. The receivers are Linux hosts on the leaves. OSPF provides unicast routing; PIM sparse mode runs on every routed interface. The configurations, scripts and raw outputs are in the lab repository. It’s a virtual lab, so it proves protocol behaviour, not hardware performance.
The figure has the addresses you’ll see in the outputs below. The group plan decides how each group is routed:
| Groups | Mode | Rendezvous point | Used for |
|---|---|---|---|
| 232.0.0.0/8 | SSM | None: receivers name the source | Feed A 232.1.1.1, feed B 232.1.2.1 |
| 239.1.0.0/16 | ASM | XR-B, 10.255.0.2 | The venue’s ASM feed |
| 239.100.0.0/16 | ASM | Anycast 10.254.1.1 on SPINE1 and SPINE2 (PIM anycast-RP) | Internal groups |
| 239.50.1.0/24 | BIDIR | Phantom 10.254.0.1, on no router | Many-to-many order bus |
| 239.200.1.1 | Not routed | None: VLAN 200 only, with IGMP snooping | Ticker plant to strategies |
Group addresses
A multicast destination address doesn’t identify a host. It identifies a group: any number of receivers, anywhere, that want the same stream. IPv4 sets aside 224.0.0.0/4 (224.0.0.0 to 239.255.255.255) for groups, and IANA carves it into ranges with different rules (RFC 5771):
| Range | Name | What it’s for |
|---|---|---|
| 224.0.0.0/24 | Local network control | Protocol traffic on one link: 224.0.0.1 all hosts, 224.0.0.2 all routers, 224.0.0.13 all PIM routers, 224.0.0.22 IGMPv3 reports. Sent with TTL 1, never routed |
| 232.0.0.0/8 | Source-specific multicast | Reserved for SSM (RFC 4607): a channel is (source, group) |
| 233.0.0.0/8 | GLOP | Group addresses derived from an AS number (RFC 3180) |
| 239.0.0.0/8 | Administratively scoped | Private use inside an organisation, like RFC 1918 for unicast (RFC 2365) |
| The rest | Globally scoped, ASM | Assigned or used by agreement |
A sender doesn’t join anything. It just sends to the group address, and the network decides who gets a copy.
Group addresses on Ethernet
A switch can’t forward on an IP address it doesn’t parse, so every group also needs an Ethernet destination. RFC 1112 defines the mapping: the fixed prefix 01:00:5e, a zero bit, then the low 23 bits of the group address. A group address has 28 significant bits (the first four are always 1110), so 5 bits are thrown away, and 32 groups share every MAC. 232.1.1.1 and 232.129.1.1 differ only in a discarded bit, so they share 01:00:5e:01:01:01 (the third column is the IP TTL the exchange sets):
On the exchange handoff (XR-A Gi0/0/0/0): destination MAC of two different groups
232.1.1.1 01:00:5e:01:01:01 16
232.129.1.1 01:00:5e:01:01:01 16
232.1.1.1 01:00:5e:01:01:01 16
232.129.1.1 01:00:5e:01:01:01 16
232.1.1.1 01:00:5e:01:01:01 16
232.129.1.1 01:00:5e:01:01:01 16
That overlap is why it matters whether a snooping switch forwards on the group’s IP address or on its MAC (NX-OS can do either), and why it’s worth avoiding groups that collide on one VLAN. To check your own groups, the multicast group to MAC tool gives the MAC for any IPv4 or IPv6 group and lists the 31 other IPv4 groups that share it.
IGMP
Hosts tell the router what they want. The Internet Group Management Protocol runs between hosts and the routers on their LAN. It has one job: tell the router which groups (and, in v3, which sources) at least one host on this link wants. The router doesn’t care which host, or how many.
There are three versions in use:
| IGMPv1 (RFC 1112) | IGMPv2 (RFC 2236) | IGMPv3 (RFC 3376) | |
|---|---|---|---|
| Join | Report to the group | Report to the group | Report to 224.0.0.22, listing sources per group |
| Leave | Silence until timeout | Leave Group to 224.0.0.2 | A report that removes the group or sources |
| Sources | No | No | Yes: include or exclude specific sources |
| Querier election | Relies on the routing protocol | Lowest IP address wins | Lowest IP address wins |
The querier
One router on each LAN is the querier. Every query interval (125 seconds by default) it sends a general query to 224.0.0.1: “does anyone still want anything?”. Each host answers for the groups it wants, after a random delay up to the maximum response time (10 seconds by default). If the robustness variable’s worth of queries (2 by default) go unanswered, the membership expires: RFC 3376 sets that group membership interval to robustness × query interval + response time, 2 × 125 + 10 = 260 seconds.
When a host leaves, it says so (IGMPv2’s Leave Group, IGMPv3’s state-change report), and the querier sends a group-specific query to check whether anyone else still wants the group before pruning it.
On the wire, an IGMPv2 join and leave, then an IGMPv3 join and leave, then the querier’s next general query:
ALGO1 eth0 (VLAN 100), ~150 s: IGMPv2 join and leave, then IGMPv3 join and leave, then waiting for a general query
1 0.000000 10.20.100.11 → 239.100.7.7 IGMPv2 46 Membership Report group 239.100.7.7
2 10.382684 10.20.100.11 → 224.0.0.2 IGMPv2 46 Leave Group 239.100.7.7
3 10.389256 10.20.100.1 → 239.100.7.7 IGMPv2 60 Membership Query, specific for group 239.100.7.7
4 12.389163 10.20.100.1 → 239.100.7.7 IGMPv3 60 Membership Query, specific for group 239.100.7.7
5 13.389562 10.20.100.1 → 239.100.7.7 IGMPv3 60 Membership Query, specific for group 239.100.7.7
6 20.777420 10.20.100.11 → 224.0.0.22 IGMPv3 58 Membership Report / Group 232.1.1.1, new source {10.10.1.10}
7 20.997378 10.20.100.11 → 224.0.0.22 IGMPv3 58 Membership Report / Group 232.1.1.1, new source {10.10.1.10}
8 30.837379 10.20.100.11 → 224.0.0.22 IGMPv3 58 Membership Report / Group 232.1.1.1, block source {10.10.1.10}
9 30.844816 10.20.100.1 → 232.1.1.1 IGMPv3 60 Membership Query, specific for group 232.1.1.1, source {10.10.1.10}
10 31.145386 10.20.100.11 → 224.0.0.22 IGMPv3 58 Membership Report / Group 232.1.1.1, block source {10.10.1.10}
11 31.152860 10.20.100.1 → 232.1.1.1 IGMPv3 60 Membership Query, specific for group 232.1.1.1, source {10.10.1.10}
12 31.844675 10.20.100.1 → 232.1.1.1 IGMPv3 60 Membership Query, specific for group 232.1.1.1, source {10.10.1.10}
13 32.155582 10.20.100.1 → 232.1.1.1 IGMPv3 60 Membership Query, specific for group 232.1.1.1, source {10.10.1.10}
14 58.176320 10.20.100.1 → 224.0.0.1 IGMPv3 60 Membership Query, general
The IGMPv2 report says only “group”, and it’s sent to the group address itself, with TTL 1 and the Router Alert option so every router on the link looks at it. The leave goes to 224.0.0.2, all routers:
IGMPv2 Membership Report, decoded
Internet Protocol Version 4, Src: 10.20.100.11, Dst: 239.100.7.7
0100 .... = Version: 4
.... 0110 = Header Length: 24 bytes (6)
Differentiated Services Field: 0x00 (DSCP: CS0, ECN: Not-ECT)
0000 00.. = Differentiated Services Codepoint: Default (0)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 32
Identification: 0x3119 (12569)
010. .... = Flags: 0x2, Don't fragment
0... .... = Reserved bit: Not set
.1.. .... = Don't fragment: Set
..0. .... = More fragments: Not set
...0 0000 0000 0000 = Fragment Offset: 0
Time to Live: 1
Protocol: IGMP (2)
Header Checksum: 0x4f34 [validation disabled]
[Header checksum status: Unverified]
Source Address: 10.20.100.11
Destination Address: 239.100.7.7
Options: (4 bytes), Router Alert
IP Option - Router Alert (4 bytes): Router shall examine packet (0)
Type: 148
1... .... = Copy on fragmentation: Yes
.00. .... = Class: Control (0)
...1 0100 = Number: Router Alert (20)
Length: 4
Router Alert: Router shall examine packet (0)
[Stream index: 0]
Internet Group Management Protocol
[IGMP Version: 2]
Type: Membership Report (0x16)
Max Resp Time: 0.0 sec (0x00)
Checksum: 0xf393 [correct]
[Checksum Status: Good]
Multicast Address: 239.100.7.7
IGMPv2 Leave Group, decoded
Internet Group Management Protocol
[IGMP Version: 2]
Type: Leave Group (0x17)
Max Resp Time: 0.0 sec (0x00)
Checksum: 0xf293 [correct]
[Checksum Status: Good]
Multicast Address: 239.100.7.7
The IGMPv3 report carries group records with source lists, sent to 224.0.0.22 rather than to the group. When a host that wasn’t listening to anything joins, that’s a state change: an Allow New Sources record for 232.1.1.1 listing 10.10.1.10, “add this source to what I want for this group”. Leaving sends the opposite, Block Old Sources:
IGMPv3 Membership Report (join), decoded
Internet Group Management Protocol
[IGMP Version: 3]
Type: Membership Report (0x22)
Reserved: 00
Checksum: 0xe4e6 [correct]
[Checksum Status: Good]
Reserved: 0000
Num Group Records: 1
Group Record : 232.1.1.1 Allow New Sources
Record Type: Allow New Sources (5)
Aux Data Len: 0
Num Src: 1
Multicast Address: 232.1.1.1
Source Address: 10.10.1.10
And the general query, from LEAF1’s SVI, the querier for VLAN 100:
General query from the querier, decoded
Internet Protocol Version 4, Src: 10.20.100.1, Dst: 224.0.0.1
0100 .... = Version: 4
.... 0110 = Header Length: 24 bytes (6)
Differentiated Services Field: 0xc0 (DSCP: CS6, ECN: Not-ECT)
1100 00.. = Differentiated Services Codepoint: Class Selector 6 (48)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 36
Identification: 0x983c (38972)
000. .... = Flags: 0x0
0... .... = Reserved bit: Not set
.0.. .... = Don't fragment: Not set
..0. .... = More fragments: Not set
...0 0000 0000 0000 = Fragment Offset: 0
Time to Live: 1
Protocol: IGMP (2)
Header Checksum: 0x3dc1 [validation disabled]
[Header checksum status: Unverified]
Source Address: 10.20.100.1
Destination Address: 224.0.0.1
Options: (4 bytes), Router Alert
IP Option - Router Alert (4 bytes): Router shall examine packet (0)
Type: 148
1... .... = Copy on fragmentation: Yes
.00. .... = Class: Control (0)
...1 0100 = Number: Router Alert (20)
Length: 4
Router Alert: Router shall examine packet (0)
[Stream index: 5]
Internet Group Management Protocol
[IGMP Version: 3]
Type: Membership Query (0x11)
Max Resp Time: 10.0 sec (0x64)
Checksum: 0xec1e [correct]
[Checksum Status: Good]
Multicast Address: 0.0.0.0
.... 0... = S: Do not suppress router side processing
.... .010 = QRV: 2
QQIC: 125
Num Src: 0
IGMP snooping
A Layer 2 switch would normally flood multicast frames to every port in the VLAN. IGMP snooping (RFC 4541) lets it listen to the IGMP conversation and forward each group only to the ports that asked for it, plus the ports where multicast routers sit. The catch is the same as for the router: memberships only stay alive if something keeps querying. In a VLAN with no PIM router there’s no querier, so you need a snooping querier on the switch. Cisco’s NX-OS guide puts it plainly: when PIM isn’t enabled on the VLAN interface, you must configure one. What a switch does with traffic for a group nobody is registered for differs: RFC 4541 says it must go to multicast router ports and may be flooded to every port.
In the lab, a VLAN with snooping and no querier: the membership is there, and then it isn’t.
LEAF1# show ip igmp snooping groups vlan 200 (t = 20 s into the feed; ALGO1 joined 3 s before it started)
200 239.200.1.1 v3 D Eth1/6
LEAF1# show ip igmp snooping groups vlan 200 (t = 150 s into the feed; ALGO1 joined 3 s before it started)
200 239.200.1.1 v3 D Eth1/6
LEAF1# show ip igmp snooping groups vlan 200 (t = 250 s into the feed; ALGO1 joined 3 s before it started)
(no entries)
LEAF1# show ip igmp snooping groups vlan 200 (t = 300 s into the feed; ALGO1 joined 3 s before it started)
(no entries)
LEAF1# show ip igmp snooping groups vlan 200 (t = 380 s into the feed; ALGO1 joined 3 s before it started)
(no entries)
PIM
Routers build the tree. IGMP gets the information to the receiver’s first router. Protocol Independent Multicast (PIM) carries it the rest of the way: between routers, it builds a distribution tree from the sources to every router that has receivers. “Protocol independent” means it has no routing protocol of its own. It reads whatever is in the unicast routing table.
PIM neighbours and the DR
PIM routers find each other with hellos sent to 224.0.0.13 every 30 seconds, with a holdtime of 105 seconds. On a LAN with several PIM routers, they elect a designated router (DR): the highest DR priority, then the highest IP address. The DR is the router that registers local sources (ASM) and sends joins for local receivers. A hello also advertises what a router can do, for example whether it supports BIDIR:
LEAF1 Ethernet1/1: PIM hello, decoded
Time to Live: 1
Protocol Independent Multicast
0010 .... = Version: 2
.... 0000 = Type: Hello (0)
Reserved byte(s): 00
Checksum: 0x14e7 [correct]
[Checksum Status: Good]
PIM Options: 4
Option 1: Hold Time: 105
Type: 1
Length: 2
Holdtime: 105
Option 19: DR Priority: 1
Type: 19
Length: 4
DR Priority: 1
Option 22: Bidirectional Capable
Type: 22
Length: 0
Option 20: Generation ID: 238795819
Type: 20
Length: 4
Generation ID: 238795819
RPF
The unicast table, read backwards. Multicast can’t use a destination to decide where to forward, so PIM turns the question around. Reverse path forwarding asks: on which interface would I send unicast traffic to the source? A multicast packet is accepted only if it arrives on that interface, the RPF interface; anything else is dropped. That one rule prevents loops, and it’s also how joins travel: a router sends its PIM join out of the RPF interface, towards the source (or the RP).
LEAF1# show ip route 10.10.1.10
IP Route Table for VRF "default"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>
10.10.1.0/24, ubest/mbest: 2/0
*via 10.0.0.8, Eth1/1, [110/81], 00:47:33, ospf-1, intra
*via 10.0.0.10, Eth1/2, [110/81], 00:47:32, ospf-1, intra
LEAF1# show ip mroute 232.1.1.1 detail
IP Multicast Routing Table for VRF "default"
Total number of routes: 3
Total number of (*,G) routes: 0
Total number of (S,G) routes: 1
Total number of (*,G-prefix) routes: 2
(10.10.1.10/32, 232.1.1.1/32), uptime: 00:01:04, ip(0) pim(0) igmp(1)
RPF-Source: 10.10.1.10 [81/110]
Data Created: No
SSM route
Nat Mode: Invalid
Nat Route Type: Invalid
UM NAT: No
Stats: 466/59648 [Packets/Bytes], 7.953 kbps
Stats: Active Flow
Incoming interface: Ethernet1/1, RPF nbr: 10.0.0.8
LISP dest context id: 0 Outgoing interface list: (count: 1) (bridge-only: 0)
Vlan100, uptime: 00:00:15, igmp
LEAF1# show ip igmp groups vlan 100
IGMP Connected Group Membership for Interface "Vlan100" - 1 total entries
Type: S - Static, D - Dynamic, L - Local, T - SSM Translated, H - Host Proxy
* - Cache Only
Group Address Type Interface Uptime Expires Last Reporter
232.1.1.1
10.10.1.10 D Vlan100 00:00:15 00:04:05 10.20.100.11
LEAF1# show ip igmp interface vlan100
IGMP Interfaces for VRF "default"
Vlan100, Interface status: protocol-up/link-up/admin-up
IP address: 10.20.100.1, IP subnet: 10.20.100.0/24
Active querier: 10.20.100.1, version: 3, next query sent in: 00:01:59
Membership count: 1
Old Membership count 0
IGMP version: 3, host version: 3
IGMP query interval: 125 secs, configured value: 125 secs
IGMP max response time: 10 secs, configured value: 10 secs
IGMP startup query interval: 31 secs, configured value: 31 secs
IGMP startup query count: 2, configured value: 2
IGMP last member mrt: 1 secs
IGMP last member query count: 2
IGMP group timeout: 260 secs, configured value: 260 secs
IGMP querier timeout: 255 secs, configured value: 255 secs
IGMP unsolicited report interval: 10 secs
IGMP robustness variable: 2, configured value: 2
IGMP reporting for link-local groups: disabled
IGMP interface enable refcount: 1
IGMP interface immediate leave: disabled
IGMP interface suppress v3-gsq: disabled
IGMP VRF name default (id 1)
IGMP Report Policy: None
IGMP Host Proxy: Disabled
IGMP Host SG Proxy: Disabled
IGMP State Limit: None
IGMP interface statistics: (only non-zero values displayed)
General (sent/received):
v1-reports: 0/0
v2-queries: 0/0, v2-reports: 0/4, v2-leaves: 0/2
v3-queries: 157/157, v3-reports: 0/133
Errors:
General Queries received with invalid destination address; v2: 0, v3: 0
Checksum errors: 0, Packet length errors: 0
Packets with Local IP as source: 0, Source subnet check failures: 0
Query from non-querier:0
Report version mismatch: 0, Query version mismatch: 0
Unknown IGMP message type: 0
Invalid v1 reports: 0
Invalid v2 reports: 0, Invalid v3 reports: 0
v3 reports with invalid auxillary length: 0
Packets dropped due to router-alert check: 0
Packets dropped due to invalid TTL: 0
Interface PIM DR: Yes
Interface vPC SVI: No
Interface vPC CFS statistics:
DR queries sent: 0
DR queries rcvd: 0
DR queries fail: 0
DR updates sent: 0
DR updates rcvd: 0
DR updates fail: 0
LEAF1# show ip pim neighbor
PIM Neighbor Status for VRF "default"
Neighbor Interface Uptime Expires DR Bidir- BFD ECMP Redirect
Priority Capable State Capable
10.0.0.8 Ethernet1/1 00:47:45 00:01:17 1 yes n/a no
10.0.0.10 Ethernet1/2 00:47:44 00:01:34 1 yes n/a no
SPINE1# show ip mroute 232.1.1.1
IP Multicast Routing Table for VRF "default"
(10.10.1.10/32, 232.1.1.1/32), uptime: 00:01:07, pim ip
Incoming interface: Ethernet1/1, RPF nbr: 10.0.0.0
Outgoing interface list: (count: 1)
Ethernet1/3, uptime: 00:00:18, pim
XR-A# show mrib route 232.1.1.1
(10.10.1.10,232.1.1.1) RPF nbr: 10.10.1.10 Flags: RPF
Up: 00:01:09
Incoming Interface List
GigabitEthernet0/0/0/0 Flags: A, Up: 00:01:09
Outgoing Interface List
GigabitEthernet0/0/0/1 Flags: F NS, Up: 00:00:20
XR-A# show pim topology 232.1.1.1
IP PIM Multicast Topology Table
Entry state: (*/S,G)[RPT/SPT] Protocol Uptime Info
Entry flags: KAT - Keep Alive Timer, AA - Assume Alive, PA - Probe Alive
RA - Really Alive, IA - Inherit Alive, LH - Last Hop
DSS - Don't Signal Sources, RR - Register Received
SR - Sending Registers, SNR - Sending Null Registers
E - MSDP External, EX - Extranet
MFA - Mofrr Active, MFP - Mofrr Primary, MFB - Mofrr Backup
DCC - Don't Check Connected, ME - MDT Encap, MD - MDT Decap
MT - Crossed Data MDT threshold, MA - Data MDT Assigned
SAJ - BGP Source Active Joined, SAR - BGP Source Active Received,
SAS - BGP Source Active Sent, IM - Inband mLDP, X - VxLAN
Interface state: Name, Uptime, Fwd, Info
Interface flags: LI - Local Interest, LD - Local Dissinterest,
II - Internal Interest, ID - Internal Dissinterest,
LH - Last Hop, AS - Assert, AB - Admin Boundary, EX - Extranet,
BGP - BGP C-Multicast Join, BP - BGP Source Active Prune,
MVS - MVPN Safi Learned, MV6S - MVPN IPv6 Safi Learned
(10.10.1.10,232.1.1.1)SPT SSM Up: 00:01:09
JP: Join(00:00:44) RPF: GigabitEthernet0/0/0/0,10.10.1.10* Flags:
GigabitEthernet0/0/0/1 00:00:20 fwd Join(00:03:12)
Everything in an mroute entry follows from that: the incoming interface is the RPF interface, the RPF neighbour is the next hop towards the source, and the outgoing interface list is where joins (or IGMP reports) came from.
Joins and prunes
Trees are built hop by hop from the receivers towards the source. A router with an interested receiver sends a PIM Join for (S,G) or (*,G) to its RPF neighbour; that router adds the interface to its outgoing list and sends its own join upstream, and so on until the join reaches the source’s router or a router already on the tree. Joins are refreshed every 60 seconds and expire after 210. A Prune removes a branch. Here’s LEAF1’s (S,G) join, sent only on the uplink its RPF check chose. The second message, 17 seconds later, isn’t a refresh: it’s the prune, after ALGO1 left the channel at 15 seconds (the IGMP side of the same run is in the SSM section) and LEAF1’s last-member queries went unanswered:
LEAF1 Ethernet1/1 (to SPINE1): PIM
1 0.000000 10.0.0.9 → 224.0.0.13 PIMv2 68 Join/Prune
3 17.061429 10.0.0.9 → 224.0.0.13 PIMv2 68 Join/Prune
Protocol Independent Multicast
0010 .... = Version: 2
.... 0011 = Type: Join/Prune (3)
Reserved byte(s): 00
Checksum: 0xd6cc [correct]
[Checksum Status: Good]
PIM Options
Upstream-neighbor: 10.0.0.8
Address Family: IPv4 (1)
Encoding Type: Native (0)
Unicast: 10.0.0.8
Reserved byte(s): 00
Num Groups: 1
Holdtime: 210
Group 0
Group 0: 232.1.1.1/32
Address Family: IPv4 (1)
Encoding Type: Native (0)
Flags: 0x00
0... .... = Bidirectional PIM: Not set
.000 000. = Reserved: 0x00
.... ...0 = Admin Scope Zone: Not set
Masklen: 32
Group: 232.1.1.1
Num Joins: 1
IP address: 10.10.1.10/32 (S)
Address Family: IPv4 (1)
Encoding Type: Native (0)
Flags: 0x04, Sparse
0000 0... = Reserved: 0x00
.... .1.. = Sparse: Set
.... ..0. = WildCard: Not set
.... ...0 = Rendezvous Point Tree: Not set
Masklen: 32
Source: 10.10.1.10
Num Prunes: 0
PIM message types
| Type | Message | Used by |
|---|---|---|
| 0 | Hello | Every PIM router: neighbours, DR election, capabilities |
| 1 | Register | ASM: the source’s DR sends data to the RP inside unicast |
| 2 | Register-Stop | ASM: the RP tells the DR to stop registering |
| 3 | Join/Prune | Building and removing branches |
| 4 | Bootstrap | BSR: RP information flooded through the domain |
| 5 | Assert | Electing a single forwarder when two routers would forward onto one LAN |
| 8 | Candidate-RP-Advertisement | BSR: routers offering to be RP |
| 10 | DF Election | BIDIR: electing the designated forwarder per link |
ASM and the rendezvous point
In Any-Source Multicast, a receiver asks for a group without knowing its sources. The routers need a place where receivers and sources can find each other: the rendezvous point (RP), one router (or one address) that every router agrees on for each group range.
It happens in four steps:
- The receiver joins the shared tree. The receiver’s router sends a (*,G) join towards the RP, hop by hop. The tree from the RP to all receivers is the shared tree, or RP tree.
- The source registers. When a source starts sending, its DR doesn’t know who’s listening, so it wraps each packet in a unicast Register message to the RP, which sends it down the shared tree. If the RP has no receivers, it answers with a Register-Stop straight away.
- The RP joins the source. With receivers waiting, the RP joins (S,G) towards the source. Once traffic arrives natively, it sends a Register-Stop, and the wrapping ends.
- The receivers switch to the source tree. The receiver’s last-hop router now knows the source address. It joins (S,G) directly towards the source (the shortest-path tree, SPT) and prunes that source off the shared tree. If nobody else on the shared tree still needs that source, the RP prunes itself off too. On NX-OS the switch happens on the first packet unless
ip pim spt-threshold infinitykeeps a group range on the shared tree.
The register on the wire, from XR-A, the source’s first-hop router, to XR-B, the RP for 239.1.0.0/16. I captured both of XR-A’s uplinks, each with its own clock, so compare times only within one interface. The decoded message is a Null-Register: after a Register-Stop, the DR keeps sending the header alone, with no data, so the RP can tell it to keep stopping or to start again:
XR-A Gi0/0/0/1 (to SPINE1): PIM other than hellos
5 17.400657 10.0.0.2 → 10.255.0.2 PIMv2 62 Register
XR-A Gi0/0/0/2 (to SPINE2): PIM other than hellos
1 0.000000 10.0.0.3 → 224.0.0.13 PIMv2 68 Join/Prune
3 12.188491 10.255.0.2 → 10.0.0.2 PIMv2 60 Register-stop
5 17.052952 10.0.0.3 → 224.0.0.13 PIMv2 68 Join/Prune
.... 0001 = Type: Register (1)
.1.. .... .... .... .... .... .... .... = Null-Register: Yes
Protocol Independent Multicast
0010 .... = Version: 2
.... 0001 = Type: Register (1)
Reserved byte(s): 00
Checksum: 0x9eff [correct]
[Checksum Status: Good]
PIM Options
Flags: 0x40000000
0... .... .... .... .... .... .... .... = Border: No
.1.. .... .... .... .... .... .... .... = Null-Register: Yes
0000 .... = IP Version: Dummy Header (0)
Dummy HeaderIPv4
Source: 10.10.1.10
Group: 239.1.12.1
During the switchover, the last-hop router holds both trees at once, the shared tree towards the RP through SPINE1 and the source tree through SPINE2:
LEAF1# show ip mroute 239.1.10.1
IP Multicast Routing Table for VRF "default"
(*, 239.1.10.1/32), uptime: 00:00:06, igmp ip pim
Incoming interface: Ethernet1/1, RPF nbr: 10.0.0.8
Outgoing interface list: (count: 1)
Vlan100, uptime: 00:00:06, igmp
(10.10.1.10/32, 239.1.10.1/32), uptime: 00:00:06, ip mrib pim
Incoming interface: Ethernet1/2, RPF nbr: 10.0.0.10
Outgoing interface list: (count: 1)
Vlan100, uptime: 00:00:06, mrib
On XR-B, the RP, the same group arrives through Decapstunnel0: the source’s packets, unwrapped from the registers, going down the shared tree:
XR-B# show mrib route 239.1.10.1
(*,239.1.10.1) RPF nbr: 10.255.0.2 Flags: C RPF
Up: 00:00:07
Incoming Interface List
Decapstunnel0 Flags: A, Up: 00:00:07
Outgoing Interface List
GigabitEthernet0/0/0/1 Flags: F NS, Up: 00:00:07
(10.10.1.10,239.1.10.1) RPF nbr: 10.255.0.2 Flags: L RPF
Up: 00:14:37
Incoming Interface List
Decapstunnel0 Flags: A, Up: 00:00:07
Finding the RP
Every router has to agree on the RP for each group range. There are three ways:
| Method | How | Notes |
|---|---|---|
| Static | ip pim rp-address on every router |
Simple and predictable; every router must be configured |
| Auto-RP | Cisco: candidate RPs announce on 224.0.1.39, a mapping agent announces the result on 224.0.1.40 | Proprietary; the announcements themselves are multicast, which needs care |
| BSR | Standard (RFC 5059): candidate RPs advertise to a bootstrap router, which floods the RP set in Bootstrap messages | Hop-by-hop flooding, no multicast dependency |
Anycast RP and MSDP
An RP is a single point of failure for new sources and new receivers. Anycast RP puts the same RP address on several routers; unicast routing sends each router’s registers and joins to the nearest one. The RPs then need to tell each other about the sources they’ve learnt, or a receiver on one RP would never hear a source registered on another. Two ways: MSDP (RFC 3618, used for anycast RP per RFC 3446), where RPs exchange source-active messages over TCP; or PIM Anycast-RP (RFC 4610), where an RP forwards each register to the other RPs in the set. NX-OS implements the second as ip pim anycast-rp, which is what the lab uses: both spines carry 10.254.1.1 and list each other as anycast-RP peers. MSDP is also how separate PIM domains share sources. Here’s SPINE1’s view of the RP for an internal group:
SPINE1# show ip pim rp 239.100.1.1
PIM RP Status Information for VRF "default"
PIM RP Information for group 239.100.1.1 in VRF "default"
RP: 10.254.1.1*, (0),
uptime: 00:29:23 priority: 255,
RP-source: (local),
group ranges:
239.100.0.0/16
SSM
Name the source. Source-Specific Multicast removes the hardest part of ASM: discovering sources. The receiver asks for a channel, (S,G), naming the source as well as the group. Its router joins (S,G) straight towards S, and data arrives on the shortest path from the first packet. No RP, no register, no shared tree, no switchover. RFC 4607 reserves 232.0.0.0/8 for it; the host side is IGMPv3 (or MLDv2 for IPv6), because only v3 can name a source (RFC 4604).
SSM works whenever receivers can learn the source address some other way, which for an exchange feed, a video channel or a known publisher, they can. Hosts that only speak IGMPv2 can still use it if the router maps their joins to a source (SSM mapping, ip igmp ssm-translate on NX-OS).
ALGO1 eth0: IGMP
1 0.000000 10.20.100.11 → 224.0.0.22 IGMPv3 58 Membership Report / Group 232.1.1.1, new source {10.10.1.10}
2 0.100004 10.20.100.11 → 224.0.0.22 IGMPv3 58 Membership Report / Group 232.1.1.1, new source {10.10.1.10}
3 15.063918 10.20.100.11 → 224.0.0.22 IGMPv3 58 Membership Report / Group 232.1.1.1, block source {10.10.1.10}
4 15.067428 10.20.100.1 → 232.1.1.1 IGMPv3 60 Membership Query, specific for group 232.1.1.1, source {10.10.1.10}
5 15.999941 10.20.100.11 → 224.0.0.22 IGMPv3 58 Membership Report / Group 232.1.1.1, block source {10.10.1.10}
6 16.003166 10.20.100.1 → 232.1.1.1 IGMPv3 60 Membership Query, specific for group 232.1.1.1, source {10.10.1.10}
7 16.068365 10.20.100.1 → 232.1.1.1 IGMPv3 60 Membership Query, specific for group 232.1.1.1, source {10.10.1.10}
8 17.004839 10.20.100.1 → 232.1.1.1 IGMPv3 60 Membership Query, specific for group 232.1.1.1, source {10.10.1.10}
Internet Protocol Version 4, Src: 10.20.100.11, Dst: 224.0.0.22
0100 .... = Version: 4
.... 0110 = Header Length: 24 bytes (6)
Differentiated Services Field: 0xc0 (DSCP: CS6, ECN: Not-ECT)
1100 00.. = Differentiated Services Codepoint: Class Selector 6 (48)
.... ..00 = Explicit Congestion Notification: Not ECN-Capable Transport (0)
Total Length: 44
Identification: 0x0000 (0)
010. .... = Flags: 0x2, Don't fragment
0... .... = Reserved bit: Not set
.1.. .... = Don't fragment: Set
..0. .... = More fragments: Not set
...0 0000 0000 0000 = Fragment Offset: 0
Time to Live: 1
Protocol: IGMP (2)
Header Checksum: 0x95d6 [validation disabled]
[Header checksum status: Unverified]
Source Address: 10.20.100.11
Destination Address: 224.0.0.22
Options: (4 bytes), Router Alert
IP Option - Router Alert (4 bytes): Router shall examine packet (0)
Type: 148
1... .... = Copy on fragmentation: Yes
.00. .... = Class: Control (0)
...1 0100 = Number: Router Alert (20)
Length: 4
Router Alert: Router shall examine packet (0)
[Stream index: 0]
Internet Group Management Protocol
[IGMP Version: 3]
Type: Membership Report (0x22)
Reserved: 00
Checksum: 0xe4e6 [correct]
[Checksum Status: Good]
Reserved: 0000
Num Group Records: 1
Group Record : 232.1.1.1 Allow New Sources
Record Type: Allow New Sources (5)
Aux Data Len: 0
Num Src: 1
Multicast Address: 232.1.1.1
Source Address: 10.10.1.10
BIDIR
One tree, many senders. Bidirectional PIM (RFC 5015) is for groups where many hosts send and receive: one (*,G) tree rooted at the RP address carries traffic in both directions, up towards the RP and down to the receivers. Sources don’t register and routers keep no (S,G) state, so state stays flat however many senders there are.
Without registers, the routers need another way to keep traffic loop-free on each link: a designated forwarder (DF), elected per link and per RP address, the router with the best route to the RP. Only the DF forwards traffic up towards the RP or down onto the link. The election uses DF Election messages (Offer, Winner, Backoff, Pass). I captured both of LEAF1’s uplinks while SPINE1, which owns the primary RP link, took it down and brought it back. On the SPINE1–LEAF1 link, SPINE1 returns with metric 0 (it’s on the RP link again), LEAF1 backs off, and SPINE1 wins. On the SPINE2–LEAF1 link, SPINE2 had been DF while its /29 was the only route to the RP; once SPINE1’s longer prefix was back, SPINE2’s metric (110/42) was worse than LEAF1’s (110/41), so it backed off and passed the role to LEAF1:
SPINE1-LEAF1 link: PIM DF election messages (time, sender, subtype, RP, route metric to the RP)
20.452 s 10.0.0.9 Winner RP 10.254.0.1 metric 110/41
21.550 s 10.0.0.8 Offer RP 10.254.0.1 metric 0/0
21.552 s 10.0.0.9 Backoff RP 10.254.0.1 metric 110/41
21.847 s 10.0.0.9 Offer RP 10.254.0.1 metric infinite
21.909 s 10.0.0.8 Offer RP 10.254.0.1 metric 0/0
22.010 s 10.0.0.8 Offer RP 10.254.0.1 metric 0/0
22.110 s 10.0.0.8 Offer RP 10.254.0.1 metric 0/0
22.151 s 10.0.0.8 Winner RP 10.254.0.1 metric 0/0
SPINE2-LEAF1 link: PIM DF election messages (time, sender, subtype, RP, route metric to the RP)
25.849 s 10.0.0.10 Winner RP 10.254.0.1 metric 0/0
27.860 s 10.0.0.11 Offer RP 10.254.0.1 metric 110/41
27.862 s 10.0.0.10 Winner RP 10.254.0.1 metric 0/0
27.881 s 10.0.0.10 Winner RP 10.254.0.1 metric 110/42
27.983 s 10.0.0.11 Offer RP 10.254.0.1 metric 110/41
27.986 s 10.0.0.10 Backoff RP 10.254.0.1 metric 110/42
28.957 s 10.0.0.10 Pass RP 10.254.0.1 metric 110/42
28.962 s 10.0.0.11 Winner RP 10.254.0.1 metric 110/41
Protocol Independent Multicast
0010 .... = Version: 2
.... 1010 = Type: DF election (10)
0010 .... = DF Subtype: DF Winner (2)
.... 0000 = DF reserved: 0
Checksum: 0xc9e0 [correct]
[Checksum Status: Good]
PIM Options
RP: 10.254.0.1
Address Family: IPv4 (1)
Encoding Type: Native (0)
Unicast: 10.254.0.1
DF Metric Preference: 0
Metric: 0
The RP address in BIDIR is just a direction, so it doesn’t have to belong to any router. In the phantom RP design it’s an unused address on a subnet, the RP link. Two routers advertise that subnet with different mask lengths; longest match makes the longer prefix primary, and the shorter one takes over if it disappears.
Where each one is used
The models aren’t competitors so much as tools for different shapes of traffic. The question that decides it is who sends, and whether the receivers know the senders in advance.
| Kind | Where you’ll find it | Why it fits |
|---|---|---|
| Link-local groups, 224.0.0.0/24 | Routing and first-hop protocols on almost every network: OSPF (224.0.0.5 and .6), EIGRP (.10), PIM (.13), VRRP (.18), mDNS (224.0.0.251) | The protocol only needs its neighbours on the link, so nothing is routed and no tree is built |
| SSM | IPTV and other broadcast video, where each channel has a known source; market-data feeds from venues that publish their source addresses | One sender, known in advance: no RP to run, and the shortest path from the first packet. RFC 4607 calls it “particularly well-suited” to this kind of dissemination |
| ASM | Applications that discover each other at run time; market-data feeds that venues deliver as groups rather than SSM channels; VXLAN flood-and-learn underlays, where each VNI’s broadcast and unknown traffic rides a multicast group | Receivers don’t have to know the sources; the RP introduces them |
| BIDIR | Many-to-many groups where every member sends as well as listens, such as a trading firm’s order or messaging bus, or the order bus in this lab | One (*,G) tree per group, whatever the number of senders, so state stays flat |
| Anycast RP or MSDP | Any ASM network that can’t depend on a single RP; MSDP also between separate PIM domains | Redundant RPs that still know every source |
Scoping
Two tools keep multicast inside the area it’s meant for:
- TTL. Every routed hop decrements it; 224.0.0.0/24 traffic is sent with TTL 1 and never crosses a router. A source can limit its reach by setting a low TTL, but that’s fragile.
- Administrative boundaries. RFC 2365 reserves 239.0.0.0/8 for scopes an organisation defines, and routers enforce them with a boundary on an interface: an access list of the groups allowed to cross, in either direction, with everything else stopped. That’s how you keep internal groups internal and accept only what you expect from a partner.
On XR-A’s handoff, the boundary permits only the contracted ranges. The uncontracted 232.9.9.9 has an (S,G) entry but no incoming interface: XR-A won’t accept it from the venue. The internal 239.100.1.1, which a host on the exchange side asked for, lists the handoff with LI (local interest) but no F: XR-A knows someone there wants it and doesn’t forward it:
XR-A# show running-config multicast-routing
multicast-routing
address-family ipv4
interface GigabitEthernet0/0/0/0
boundary EXCH-A-CONTRACT
!
interface all enable
!
!
XR-A# show mrib route 232.9.9.9
(10.10.1.10,232.9.9.9) RPF nbr: 10.10.1.10 Flags: RPF
Up: 00:00:15
Outgoing Interface List
GigabitEthernet0/0/0/1 Flags: F NS, Up: 00:00:15
XR-A# show mrib route 239.100.1.1
(*,239.100.1.1) RPF nbr: 10.0.0.3 Flags: C RPF
Up: 00:00:16
Incoming Interface List
GigabitEthernet0/0/0/2 Flags: A NS, Up: 00:00:16
Outgoing Interface List
GigabitEthernet0/0/0/0 Flags: LI, Up: 00:00:16
(10.20.30.11,239.100.1.1) RPF nbr: 10.0.0.3 Flags: RPF
Up: 00:01:30
Incoming Interface List
GigabitEthernet0/0/0/2 Flags: A, Up: 00:01:30
Where to go next
This is the machinery. A trading day in multicast, the companion to this article, puts it under load: A/B feeds and arbitration, SSM against ASM for exchange feeds, BIDIR for an order bus, the extranet boundary, querier failures and microbursts, all measured in the same lab.
References
- RFC 1112, Host extensions for IP multicasting: IGMPv1 and the Ethernet mapping (section 6.4), recorded by IANA as the 01-00-5E multicast MAC range; RFC 2236, IGMPv2; RFC 3376, IGMPv3: query interval, robustness, group membership interval; RFC 4604, IGMPv3 and MLDv2 for SSM.
- RFC 5771, IPv4 multicast address assignments, and the IANA IPv4 multicast address registry for the link-local groups (OSPF, EIGRP, PIM, VRRP, mDNS); RFC 2365, administratively scoped IP multicast; RFC 3180, GLOP addressing.
- RFC 4541, IGMP and MLD snooping switches.
- RFC 7761, PIM Sparse Mode: hellos, DR election, RPF, joins and prunes, registers, SPT switchover; RFC 4607, Source-Specific Multicast; RFC 5015, Bidirectional PIM; RFC 5059, Bootstrap Router.
- RFC 3618, MSDP; RFC 3446, Anycast RP using PIM and MSDP; RFC 4610, Anycast-RP using PIM.
- Cisco, VXLAN data plane learning (flood and learn) lab: a PIM ASM underlay with anycast RP carrying each VNI’s broadcast and unknown traffic.
- Cisco, Nexus 9000 NX-OS Multicast Routing Configuration Guide 10.6(x): Configuring PIM and PIM6 and Configuring IGMP Snooping.
- Multicast group to MAC tool: the mapping for any group, and every group that shares its MAC.
- A trading day in multicast: the same mechanisms under load, measured.
- Lab code, configurations and raw results: networkfactory-labs.
Configurations are illustrative: validate them on your own platforms and software versions in a lab before production use. Cisco, IOS-XR, NX-OS and Nexus are trademarks of Cisco Systems, Inc.