Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extended queries optional #899

Merged
merged 111 commits into from Aug 28, 2019

Conversation

@pm47
Copy link
Member

commented Mar 14, 2019

This is the implementation of lightningnetwork/lightning-rfc#557.

sstone and others added 30 commits Sep 10, 2018
Add extended query messages
Extended query messages are allow peers to exchange short channel id and timestamps, and
to query either announcement + updates or just updates:

- query_channel_range_ex is used asked to ask for a list of (short channel ids + timestamp)
- reply_channel_range_ex will return a list of (short channel ids + timestamp)
- query_short_channel_ids_ex is used to ask for channel updates and optionaly the matching channel announcement
- query_short_channel_ids_end_ex is sent when an extended query has been completed
Add a feature bit for extended channel range queries
Last feature bit is currently 7, we use bit 15 to not interfere with new
features being added to the LN spec.
Correctly handle multiple channel_range_replies
The scheme we use to keep tracks of channel queries with each peer would forget about
missing data when several channel_range_replies are sent back for a single channel_range_queries.
RoutingSync: remove peer entry properly
* Remove peer entry on our sync map only when we've received
a `reply_short_channel_ids_end` message.
* Make routing sync test more explicit
Router: clean our sync state when we (re)connect to a peer
We must clean up leftovers for the previous session and start the sync process again.
Router: reset sync state on reconnection
When we're reconnected to a peer we will start a new sync process and should reset our sync
state with that peer.
add extended query flag to our log message
It will tell us if sender requests channel announcements and updates or just updates
Router: implement new extended queries
- reply_channel_range include  a list of channel id + both channel update timestamp
- query_short_channl_ids include a list of channel id + flag, where flag specifies what the receiver
should send back (a combination of channel annoucement, channel updated #1 and channel update #2)
araspitzu and others added 4 commits Aug 7, 2019
Typed amounts (#1088)
* Type all amounts used in eclair

* Add eclair.MilliSatoshi class

* Use bitcoin-lib 0.14

* Add specialized codecs for Satoshi/MilliSatoshi

* Rename 'toSatoshi' to 'truncateToSatoshi' to highlight it's a precision-losing conversion
Extended Queries: use TLV format for optional data (#1072)
* Extended Queries: use TLV format for optional data

Optional query extensions now use TLV instead of a custom format.
Flags are encoded as varint instead of bytes as originally proposed. With the current proposal they will all fit on a single byte, but will be
much easier to extends this way.

* Move query message TLVs to their own namespace

We add one new class for each TLV type, with specific TLV types, and encapsulate codecs.

* Optional TLVs are represented as a list, not an optional list

TLVs that extend regular LN messages can be represented as a TlvStream and not an Option[TlvStream] since we don't need
to explicitely terminate the stream (either by preprending its length or using a specific terminator) as we do in Onion TLVs.

No TLVs simply means that the TLV stream is empty.

* Update to match  BOLT PR

Checksums in ReplyChannelRange now have the same encoding as short channel ids and timestamps: one byte for
the encoding type (uncompressed or zlib) followed by encoded data.

* TLV Stream: Implement a generic "get" method for TLV fields

If a have a TLV stream of type MyTLV which is a subtype of TLV, and MyTLV1 and MYTLV2 are both
subtypes of MyTLV then we can use stream.get[MyTLV1] to get the TLV record of type MYTLV1 (if any)
in our TLV stream.

* Extended range queries: Implement latest BOLT changes

Checksums are just transmitted as a raw array, with optional compression as it would be useless here.

* Use extended range queries on regtest and testnet

We will use them on mainnet as soon as lightningnetwork/lightning-rfc#557 has been merged.

* Address review comments

* Router: rework handling of ReplyChannelRange

We remove the ugly and inefficient zipWithIndex we had before

* NodeParams: move fee base check to its proper place

* Router: minor cleanup
@pm47

This comment has been minimized.

Copy link
Member Author

commented Aug 22, 2019

@sstone seems like includeNode1 and includeNode2 are not implemented?

BTW there is a related bug in the current implementation:

The receiver [of a query_short_channel_ids message]:
MUST follow with any node_announcements for each channel_announcement

sstone and others added 10 commits Aug 26, 2019
Channel range queries: send back node announcements (#1108)
* Channel Range Queries: send back node announcements if requested

This PR adds support for sending back node announcements when replying to channel range queries:
- when explicitly requested (bit is set in the optional query flag)
- when query flags are not used and a channel announcement is sent (as per the BOLTs)

A new configuration option `request-node-announcements` has been added in the `router` section. If set to true, we
will request node announcements when we receive a channel id (through channel range queries) that we don't know of.
This is a setting that we will probably turn off on mobile devices.

* Increase tests timeouts

There is now more work to do.

* Test query sync with and without node announcements

* Router: minor fix

* Router: rework query handling
pm47 added 4 commits Aug 28, 2019
pm47 added 4 commits Aug 28, 2019
@sstone
sstone approved these changes Aug 28, 2019

@pm47 pm47 merged commit 2f42538 into master Aug 28, 2019

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
continuous-integration/travis-ci/push The Travis CI build passed
Details

@pm47 pm47 deleted the extended-queries-optional branch Aug 28, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.