Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] BOLT 7: Inventory-based gossip #584

Open
wants to merge 6 commits into
base: master
from

Conversation

Projects
None yet
4 participants
@sstone
Copy link
Collaborator

sstone commented Feb 28, 2019

This is a proposal for adding inventory messages to the gossip protocol. With the current design, nodes which receive gossip messages (node announcements, channel announcements and updates) will relay them to their peers, which will result in duplicates for nodes that are connected to many different peers.

Nodes which support the option_inv_gossip feature will instead broadcast inventory messages which contain identifiers for gossip messages they've received. Receiving node will compare these identifiers to their local view of the routing table, and ask for missing or outdated messages using channel queries. This implies that option_inv_gossip cannot be used without gossip_queries.

It builds upon PR #571 (unification of feature bits, which includes a definition for option_inv_gossip) and #557 (extended channel queries).
More specifically, it is a "companion" PR to #557:

  • extended channel queries are used to efficiently synchronise routing tables between a node that is often offline and a very limited number of peers
  • inventory-based gossip is used to minimize duplicate gossip traffic for nodes that are connected to many different peers
  • they share similar concern: how to efficiently advertise information about channel announcement and updates

In fact, the inventory message that I propose to use here is almost identical to an extended reply_channel_range message and includes short channel ids, checksums and timestamps, to allow receiving to efficiently query messages based on content (checksum) and timestamps.

Node announcements have been left out on purpose because they have a much more limited impact on bandwidth than channel updates, and because gains would be smaller (you would still need to advertise node ids which are 33 bytes long).

sstone added some commits Jan 22, 2019

BOLT7: extend channel range queries with optional fields
Nodes that support extended queries will append an additional extended query flag to
their `query_channel_range` queries. If the receiver supports extended queries and
understands this flag, it will append the required additional data to its
`reply_channel_range` message.

There is currently only one type of additional data: one timestamp and one checksum
per `channel_update`.
The checksum is a simple Adler32 checksum computed over the `channel_update`
with `timestamp` and `signature` omitted.
Together they can be used to avoid querying `channel_updates` that are older than
the ones you already have, or that are newer but don't include new information.

Nodes can then append additional data to their `query_short_channel_ids` messages, which
consists in one flag per short channel id and specifies what they would like to
receive (`channel_announcement`, or/and one `channel_update` or both`).
BOLT 7: optional inventory-based gossip
Inventory messages is an optional feature that is used to reduce duplicate gossip traffic.
Instead of sending `channel_annoucement` or `channel_update` messages, nodes will send an
inventory message to peers that support this option.

Inventory messages are similar to channel queries and contain a list of short channel ids, checksums
and timestamps. Receiving nodes will check inventory data and use channel queries to retrieve missing
or outdated channel announcements and updates.

@sstone sstone added the 2019-03-04 label Mar 2, 2019

@rustyrussell
Copy link
Collaborator

rustyrussell left a comment

I'm going to push a small formatting fixup (we generate from the spec, so this makes it much easier to implement!)

### The `query_short_channel_ids`/`reply_short_channel_ids_end` Messages
1. type: 261 (`query_short_channel_ids`) (`gossip_queries`)
2. data:
* [`32`:`chain_hash`]
* [`2`:`len`]
* [`len`:`encoded_short_ids`]
* [`2`:`option_len`]
* [`len`:`option_encoded_query_flags`]

This comment has been minimized.

@rustyrussell

rustyrussell Mar 18, 2019

Collaborator

len needs to be option_len here.

@@ -636,6 +656,9 @@ timeouts. It also causes a natural ratelimiting of queries.
* [`1`:`complete`]
* [`2`:`len`]
* [`len`:`encoded_short_ids`]
* [`1`:`option_extended_query_flag`]
* [`2`:`option_extended_info_len`]
* [`len`:`option_extended_info`]

This comment has been minimized.

@rustyrussell

rustyrussell Mar 18, 2019

Collaborator

Here too, len -> option_extended_info_len.

rustyrussell added some commits Mar 18, 2019

BOLT 7: Make up option names for 9004f9c
Optional fields need an option name to be autogenerated correctly (we
also need to support the older versions without these fields).

Also, `len` was misused where the field name was something else.

Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
BOLT 7: fix for b7df807
Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>

@rustyrussell rustyrussell self-requested a review Mar 18, 2019

@n1bor

This comment has been minimized.

Copy link

n1bor commented Mar 20, 2019

Other thought was to get nodes to opt-out of updates for most channels from most peers. Since node has a full map of the network it could tell most peers not to send updates for a list of channels that it knows it will get quicker from multiple other peers.
Some info here: ACINQ/eclair#864

@pm47

This comment has been minimized.

Copy link
Collaborator

pm47 commented Mar 20, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.