Ty Everett (ty@projectbabbage.com)
MLD (Multicast Listener Discovery) for IPv6 provides a way for a host to indicate to an upstream router that it is interested in receiving multicast traffic from a particular group. While this is useful in some cases (such as when miners in Bitcoin want to propagate blocks and subtrees to one another efficiently), it does not solve every use-case. This document will explore the limitations of existing IPv6 multicast protocols like MLD, both with respect to the broadcast of transactions to miners and overlays, as well as the acquisition of transaction status updates like merkle proofs or double-spend attempt notifications from network operators. We will briefly discuss the economic incentives of existing multicast protocols, how they could be improved with Bitcoin, and we will conclude by defining a set of general requirements and constraints that a potential new, Bitcoin-oriented IPv6 multicast protocol ought to follow.
MLD enables a connected device to send a router a "report" about the multicast traffic that it would like to receive over a directly connected link. When the router sees incoming traffic destined for the multicast group the connected device cares about, it will forward the traffic to the device.
This is useful for a few reasons:
- Many devices can care about the same multicast group
- The sender can send the traffic once to the router at the group address
- Then, the router lets the traffic pass to all parties that care about it
This makes it extremely efficient for senders to send things to multiple recipients (i.e. multicast). The router only needs to let its upstream routers know it cares about a particular multicast group if any of its downstream connected devices also care about that traffic.
This, in itself, has some useful applications in Bitcoin: for example, suppose a group of miners want a fast way to propagate newly-solved blocks to one another. The solver could send the proof of his solution to a multicast group address, and it would immediately be propagated to the other nodes who are listening. Thus, they avoid having to send individual copies to each person, saving valuable time. These savings compound as the size of the data increases; propagation of incoming transactions, new subtrees, and other information is also possible in the same way.
However, massive scale demands the use of multicast not only within one single group between miners, but also among potentially trillions of groups, to ensure propagation of transactions. Conventional IPv6 multicast protocols like MLD are oriented around managing individual groups, and there's no network-layer filtering mechanism to ensure applications only receive the data they need.
In short, we can't just use a single multicast group for everything and flood all the participants in the network. We need to dynamically support the delivery of transaction-specific data (broadcasts, merkle proofs, double-spend notifications) to all parties who need it, something with the right primitives and assumptions built from the ground up for Bitcoin, and something with the right economic incentives as well.
When a transaction is broadcast, it's important that nodes can be made aware of it. Nodes have an interest in receiving transactions around the same time, and ensuring transactions are not lost. It therefore makes sense to establish a multicast group of some sort for transaction broadcast, where nodes can subscribe to new transactions. It's merely a question of how this group is to be set up.
Notably, different parties have different interests in various transactions. The version number of a transaction provides a way to identify the nature of the transaction, and can be used in various systems built on top of Bitcoin. If these systems had a way to filter transactions by version, and only receive transactions of a particular one, they would be able to avoid processing unnecessary data.
There have also been proposed attack vectors in which a sender broadcasts a transaction to the miners for inclusion in a block without notifying overlay nodes. This would bring the overlay nodes out-of-sync with the miners' view of the network. When a multicast group is used for transaction broadcast, in which the overlay nodes are notified automatically, this type of attack becomes far more difficult.
Therefore, it makes sense to establish at least one multicast group for each possible transaction version. Transaction version numbers are four bytes, meaning a total of at least 2^32 multicast groups. Overlay nodes listen on the specific ones they care about, and the miners listen across all of them. However, in the name of scalability we can go further, and as we will see later there are other practical reasons to do so as well.
A transaction ID comprises a 256-bit hash. The transaction ID is known at the time of broadcast, making it useful for miners and overlay providers wishing to parallelize their transaction ingest capabilities across many machines. This is to say, a portion of the TXID can be incorporated into the multicast group address used for transaction broadcast.
In a protocol where overlay node operators were able to stipulate for which blocks of multicast group addresses they would like traffic (as opposed to just a single one), they would be able to not only subscribe to transactions of any particular version, but also split this firehose into acceptable volumes that could be processed in parallel by different machines at their discretion. Meanwhile, miners who need to process inbound transactions across all version numbers would be able to further parallelize this process if desired. With the multicast group prefix and scheme widely known, these entities could establish many points of presence across geographies.
However, despite these clear benefits, having specific multicast groups for relatively small numbers of transactions is most useful not for transaction broadcast but for proof acquisition and other updates from network operators. Individual users, the originators of transactions, don't want to waste bandwidth processing these updates for large numbers of transactions they don't care about. Still, most transactions generally have at least two concerned parties, all of whom will be interested in knowing about their status and eventual merkle proof. It is valuable when miners, overlay network providers, and other entities are able to significantly reduce the costs associated with providing this type of information. The most efficient means for them to do so is through multicast, transmitting it one time for all who have subscribed to receive it.
Overlay nodes, who generally care about these updates for all transactions of a specific version, are able to subscribe to the multicast group address block associated with updates for all transactions of the version(s) for which they are interested. Notably, if this firehose of merkle proofs and other transaction updates is large, they can choose to split the load across many machines, as they do for transaction ingest. Since a portion of the TXID of each transaction is included in the transaction updates multicast group address after the 32 version bits, the load can be split however desired by varying each machine's prefix length for which updates are requested.
The next two sections of this document will delve into further details about the two aformentioned use-cases of the new Bitcoin IPv6 multicast protocol: transaction ingest (broadcast to miners and overlays) and transaction updates (like acquiring merkle proofs).
A multicast protocol that supported "reports" by listeners similar to MLD, but where these reports could encompass an entire block of multicast groups, would enable significant improvements in the way transactions are broadcast, both to miners and to overlay nodes.
Consider, as part of the protocol definition, that all parties listeners (miners, overlay nodes) and broadcasters (network users) agree on a set of IPv6 addresses to use for multicast. We define a scheme like the following:
2602:f9f8:0000:xxxx:xxxx:yyyy:yyyy:yyyy
Where xxxx:xxxx denotes the 32 bits used for the transaction version number, and yyyy:yyyy:yyyy denotes the first 48 bits of the TXID of the transaction (note that full IPv6 addresses will be used throughout this document for extra clarity).
When all parties have agreed about this scheme, we can proceed as follows:
- When a listener joins the network, they subscribe for the address block denoting the transactions they want:
- Miners (in aggregate, across all their machines) subscribe to all transaction broadcasts across all versions:
2602:f9f8::/48 - Overlay nodes (in aggregate) generally only subscribe to a single version. For example, version
5:2602:f9f8:0000:0000:0005::/80 - A specific machine operated by a high-volume overlay might be responsible for ingesting, say, a sixteenth of the total volume:
2602:f9f8:0000:0000:0005:5000::/84or2602:f9f8:0000:0000:0005:9000::/84
- Miners (in aggregate, across all their machines) subscribe to all transaction broadcasts across all versions:
- The listener's ISPs notify other ISPs of their interest in packets destined for these addresses
- Potentially, in the case of overlays or systems that are private, the listener's report might be constrained to particular source (broadcaster) addresses
- When a sender (network user) broadcasts a transaction, they send it to their ISP with the correct multicast group destination address
- This corresponds to the appropriate version number of the transaction, as well as the first 48 bits of the TXID
- The correctness of this information can easily be verified, eventually directly within network switching hardware
- All ISPs receiving the transaction route it to any subscribed listeners (miners and overlays), as it traverses their networks
- This ensures that, as long as all interconnecting ISPs support the protocol, transactions will be delivered to all parties who have subscribed to receive them
Next, we will describe a similar protocol that works in the reverse direction.
Proof acquisition has long been a difficult problem within the Bitcoin ecosystem. Miners have struggled with the costly burden of disseminating proofs to anyone who asks, while users (including, in this context, overlay node operators who rely on the SPV (Simplified Payment Verification) architecture) have often failed to acquire them when they are needed.
In the same way that broadcasts can be received by multiple miners and overlay providers when transactions are created, updates can be provided to all who have subscribed. Notably, there can be multiple providers of updates, and all of them will be received by listeners. A similar addressing scheme is agreed between the parties, as before:
2602:f9f8:0001:xxxx:xxxx:yyyy:yyyy:yyyy
- When a listener joins the network, they already know what transactions they are interested in hearing updates about:
- If the listener is a user, they likely only care about a single TXID, or a small number of them:
2602:f9f8:0001:0000:0005:dead:beef:2024/128 - If the listener is an overlay node, they likely only care about transactions of a specific version, or a small number of them:
2602:f9f8:0001:0000:0005::/80 - As before, an overlay node can split the updates across multiple machines for improved scalability and parallelization:
2602:f9f8:0001:0000:0005:5000::/84or2602:f9f8:0001:0000:0005:9000::/84
- If the listener is a user, they likely only care about a single TXID, or a small number of them:
- As before, the ISPs forward the traffic to all appropriate destinations
- When a miner wishes to make an update about a transaction, it might be any of the following:
- Acknowledging receipt of a broadcast, potentially with a digitally signed message
- Declaring the transaction has received a subtree inclusion
- Providing a merkle proof for a mined transaction
- Notifying about an orphan, and therefore a change in the merkle proof
- Notifying about an attempted double spend
- Notifying about the freezing of relevant UTXOs
- Any other type of update relevant to the transaction
- The miner sends the update to the multicast group address associated with the transaction
- Anyone who has subscribed will receive the updates
We have described the broadcast and acquisition of updates about a transaction. The next section will cover the necessary economic incentives to make this system viable.
Bandwidth is expensive, and we propose to enable multicast subscriptions over vast amounts of data. To ensure that malicious actors aren't able to flood the network with denial-of-service attacks, we need a way to prevent unnecessary subscriptions while still ensuring the system is available to the public.
Solutions in this area will likely revolve around the exchange of payments between the upstream providers and downstream consumers of multicast services. For example, we could use a mechanism similar to that which was crudely defined within BRC-29 for users to exchange a peer-to-peer Bitcoin payment as part of negotiating their subscriptions.
For miners, they have an interest in connecting with the largest sources of new transactions. They will likely find themselves negotiating with their ISPs in order to connect them with these systems, even when bandwidth is high, at significant cost. Overlay nodes, who provide value-added services, will also fall into this category, though the costs will be lower. These entities will likely engage in service-level agreements to facilitate reliable transaction and update delivery.
For users, the focus is more on small, one-off updates about the transactions they care about. These costs will be significantly less on a per-user basis. The multicast protocols could evolve to support native payments directly with Bitcoin to the ISPs based on the updates provided, or ISPs may choose to socialize these costs across all their customers.
ISPs have an interest in keeping their paying customers connected to the data they need. Generally, they may either pay or get paid by one another to send and receive the multicast traffic they need for those who listen through their networks. If a customer starts listening, the ISP must find an upstream provider and start listening there on their behalf.
A standardized protocol, in which all listeners (whether users, overlay nodes, miners, or ISPs) negotiate with their upstream routers a price in Bitcoin for packets relayed, and then maintain a positive balance with their router throughout the course of all their interactions, would greatly simplify the deployment of the system. Parties who want large volumes pay high amounts, and their ISPs will then use this money to source the data they require at the lowest cost.
For the reasons outlined earlier in this document, MLD is not suitable for these applications. However, we can use the information that has been discussed to set some ground rules which a Bitcoin multicast protocol would probably need to follow:
- So as to avoid flooding the network, it must support subscribing to entire blocks of multicast groups, not just a single group
- So as to be publicly accessible on the internet, it must reserve and declare a large block of global, publicly-routable IPv6 address space for its use
- So that overlays need not process all transactions broadcast to the miners, it must incorporate the 32 transaction version bits into multicast group addresses where broadcasts are made
- To provide for additional scalability and parallelization, it should also incorporate a portion of the TXID of each sent transaction into multicast group addresses where broadcasts are made
- So that miners need not send out multiple copies of every merkle proof and other transaction updates, it must specify a single multicast group where proofs are sent for everyone to consume
- So that consumers of merkle proofs need not subscribe to the entire firehose of updates across all transactions, it must incorporate the 32 transaction version bits into multicast group addresses where updates are sent
- For the same reason, it must incorporate a portion of the TXID into the multicast group address where updates are sent
- To mitigate denial of service attacks and high bandwidth costs, it must prevent malicious actors from subscribing to large numbers of unnecessary transaction broadcasts or transaction updates.
The core of the Bitcoin network will adopt IPv6 multicast for scalability much faster than the edges, and the rest of the Internet. As these protocols are developed, services that facilitate transaction broadcast, acquire proofs and updates, and connect to legacy v4 services will need to be developed in parallel. This document has laid the foundation for IPv6 multicast on Bitcoin, but perhaps it is the understatement of a decade to say that more work still needs to be done.
But go forth. Convince the People. Do the work. It will serve you well.