-
Notifications
You must be signed in to change notification settings - Fork 357
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for SCID Aliases #1311
Support for SCID Aliases #1311
Conversation
9778520
to
a9125e3
Compare
Codecov Report
@@ Coverage Diff @@
## main #1311 +/- ##
==========================================
+ Coverage 90.57% 90.60% +0.02%
==========================================
Files 72 73 +1
Lines 40082 40219 +137
==========================================
+ Hits 36306 36442 +136
- Misses 3776 3777 +1
Continue to review full report at Codecov.
|
4c5453a
to
b4fc67c
Compare
Given this concern, could we make a |
Hmm, we could make a newtype for it, I don't really have a strong feeling either way there, but I don't think we should make an enum, because there's not a ton (or any?) of places where we both (a) care which type it is, but (b) want to mix the two. In public APIs we mostly expose both and let the user figure it out, in internal stuff we mostly mix them and don't care which type it is. |
Was thinking a struct that has an optional alias (aliases?) or such, with some sort of accessors. Haven't stared at the PR long enough to figure what interface would be best, though. Could just be simple ones just so it's explicit which is being used. Or something more based on the use case / strongly typed to let the compiler enforce the right behavior. |
Right, I think we need to be explicit about where we want it - external accessor(s) in |
The reason for using it internally is to avoid programmatic mistakes (both now and during further changes) and the need for reviewers to meticulously examine each use of |
Right, I think my point is that there isn't a good way to do it for internal stuff that isn't needless boilerplate that adds anything - there is, I believe, only exactly one place where both (a) we co-mingle the two types and (b) we care about knowing which type the thing is. |
For public use, there's a few places now in eg lightning-invoice where we try to fetch the SCID alias and then fall back to SCID if there isn't one, that could be a util method on |
5bab80f
to
3e4494d
Compare
Yeah, I think you're right. Took a more thorough look through the PR and spec change. This just seemed rather ominous on first read without fully grasping the spec change:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Took an overall pass after better understanding the spec change. Wish there were a way to not publish nits so I can comment on them when I see them without publishing them. 😛
lightning/src/ln/channel.rs
Outdated
// If they have sent updated points, funding_locked is always supposed to match | ||
// their "first" point, which we re-derive here. | ||
self.commitment_secrets.get_secret(INITIAL_COMMITMENT_NUMBER - 1) | ||
.map(|secret| SecretKey::from_slice(&secret).ok()).flatten() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use and_then
instead of map
and flatten
. Although, do we expect from_slice
to ever fail here? If it does, the error message below isn't entirely accurate.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actaully, we can unwrap every step here, we should have already validated the whole thing by the time we get here.
lightning/src/ln/channel.rs
Outdated
/// Allowed in any state (including after shutdown) | ||
pub fn get_latest_inbound_scid_alias(&self) -> Option<u64> { | ||
self.latest_inbound_scid_alias | ||
} | ||
|
||
/// Allowed in any state (including after shutdown) | ||
pub fn get_outbound_scid_alias(&self) -> u64 { | ||
self.outbound_scid_alias | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nit: I know we haven't been consistent but can we drop the get_
prefixes?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I mean I guess, what's the rationale, though? I figured this is one of those things that everyone is gonna have an opinion on and neither is really more or less readable, but maybe you have some specific criteria in mind that makes things clearer? At least personally I like the get_
prefix as its pretty clear what's going on when you see it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Mostly to align with idiomatic rust. Maybe not so important for internal structs.
0ffca39
to
9c028f9
Compare
Rebased. |
9c028f9
to
7d7d477
Compare
if let Some(short_id) = channel.get_short_channel_id() { | ||
channel_state.short_to_id.remove(&short_id); | ||
} | ||
let mut channel = remove_channel!(self, channel_state, chan); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
About 100 lines up and maybe elsewhere, I see a call to get_channel_update_for_unicast(..).unwrap()
that seems like it may panic due to get_short_chan_id
returning None
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure that this is an issue in this PR? Its an issue once we go to 0conf, but we don't change the semantics of SCIDs here.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That's fair. On a semi-related note, I'm not sure we're compliant with this part of the spec:
- if it returns a `channel_update`:
- MUST set `short_channel_id` to the `short_channel_id` used by the incoming onion.
seems we'll always use chan.get_short_channel_id()
in channel updates on onion error.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch. This is really nasty to resolve, so I may end up suggesting we leave this for the next PR as well, but I'll work on it and open up that PR today or so.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
See #1351.
lightning/src/ln/channelmanager.rs
Outdated
@@ -3454,7 +3530,7 @@ impl<Signer: Sign, M: Deref, T: Deref, K: Deref, F: Deref, L: Deref> ChannelMana | |||
let ret_err = match res { | |||
Ok(Some((update_fee, commitment_signed, monitor_update))) => { | |||
if let Err(e) = self.chain_monitor.update_channel(chan.get_funding_txo().unwrap(), monitor_update) { | |||
let (res, drop) = handle_monitor_err!(self, e, short_to_id, chan, RAACommitmentOrder::CommitmentFirst, false, true, Vec::new(), Vec::new(), Vec::new(), chan_id); | |||
let (res, drop) = handle_monitor_err!(self, e, short_to_id, chan, RAACommitmentOrder::CommitmentFirst, chan_id, commitment_update_only); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
200 lines down, sorry -- I'm wondering if there's a possibility we'll generate Payment*Failed
events with the inbound_scid_alias
instead of the real short_channel_id
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, can you give a specific line number in reference to the top commit of the PR? line ~3730 is just short_channel_id: None
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Oops, I meant to refer to the other two places we generate PaymentPathFailed
. In either case, is it possible that our own channel failed and the alias will be used as the scid
? If so, those docs could use updating as well
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmmm, dunno how that'd happen, we get the SCID from onion_utils::process_onion_failure
which pulls from the route. I suppose its possible someone used an alias that is also one of our channels, if that's what you're asking. Or are you just saying it'd be bad if it were an alias and we'd punish the wrong channel if someone picks an alias that is also a real channel (but not the one we used)? I beileve we check that the channel's source pubkey matches the one from the route, so hopefully that is also impossible.
// on-chain transaction is ours. | ||
// We only bother storing the most recent SCID alias at any time, though our counterparty has | ||
// to store all of them. | ||
latest_inbound_scid_alias: Option<u64>, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it'd be more readable/less error-prone if we renamed inbound_scid_alias
to route_hint_scid_alias
if that's the only place we use it. Then the other could just be scid_alias
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, that isn't the only place we use them post-0conf - they're general aliases, and someone could totally use them to route over a channel prior to confirmation.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought about this a bit more and I still kinda like inbound
as the terminology here - it matches all the ways I'd think about it (its a value we received from our counterparty, and its also the value used for payments routed inbound towards us). I still kinda want to avoid terming it "route hint" cause users may use it outside of explicit invoice contexts (eg by handing someone a last-hop explicitly via some other protocol) and "route hint" sounds very invoice-y, even though the alternative uses I can see immediately are all similar concepts.
I guess I'm not quite sure why the inbound/outbound split is confusing - it seems to line up with all the things I can think of.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sg, I guess not that it's confusing per se but I thought the rename would make it more obvious that the correct alias is being used in a given context. Fine to let it rest
// i.e. can be used for inbound payments and provided in invoices, but is not used | ||
// when routing outbound payments. | ||
self.latest_inbound_scid_alias = Some(scid_alias); | ||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Given this spec requirement: MUST set alias to a value not related to the real short_channel_id.
should we return an error in the else
case here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, probably not worth closing the channel for?
@@ -6202,6 +6226,8 @@ impl<'a, Signer: Sign, K: Deref> ReadableArgs<(&'a K, u32)> for Channel<Signer> | |||
workaround_lnd_bug_4006: None, | |||
|
|||
latest_inbound_scid_alias, | |||
// Later in the ChannelManager deserialization phase we scan for channels and assign scid aliases if its missing |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would it be feasible to add coverage for this deserialization phase, maybe in an existing test?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmmm, we don't currently do anything related to checking serialization from old versions, its definitely something we should do, but seems like a substantial project. cc #1089.
7d7d477
to
55cccb5
Compare
c64afdc
to
f00ea15
Compare
This makes tests slightly more realistic by delivering `channel_update`s to `ChannelManager`s, ensuring we have forwarding data stored locally for all channels, including public ones.
f00ea15
to
21562c0
Compare
Addressed feedback, had to push a new cleanup commit at the start to deliver |
@TheBlueMatt one comment here #1311 (comment) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Took another pass though. Remaining comments are fairly minor. Happy to see this go through and take a look at the next step.
pub fn create_unannounced_chan_between_nodes_with_value<'a, 'b, 'c, 'd>(nodes: &'a Vec<Node<'b, 'c, 'd>>, a: usize, b: usize, channel_value: u64, push_msat: u64, a_flags: InitFeatures, b_flags: InitFeatures) -> (msgs::FundingLocked, Transaction) { | ||
let mut no_announce_cfg = test_default_channel_config(); | ||
no_announce_cfg.channel_options.announced_channel = false; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Getting some deja vu from another PR. Is there a way to reuse some existing utility code such that the code that follows these lines isn't needed?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, all the existing utilities end up either calling create_channel
itself or expecting to exchange AnnouncementSignatures
at the end. I could rejigger the utilities above to separate those bits out, if you prefer? I don't really have a strong feeling between that vs having some code duplication in utilities right around each other.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Don't feel too strongly. I think I already asked another contributor to do the same in another PR, heh.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hmm, on second thought I'm kinda struggling to see how to do this - the top 7 lines could be split out into three lines and four lines, but that doesn't seem so useful. Then there's the funding generation, which almost can be combined, except that we rely on exchanging the funding_locked messages in order so that we only get one channel_update
out at a time - we could swap for the create_chan_between_nodes_with_value_confirm_first
helper, but then we're stick peeling apart multiple pending messages which takes more LoC than we save :(.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No problem with passing on this. My concern was more with code repetition resulting in needing to modify two places rather than reducing lines of code.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yea, I agree, its unclear how to resolve it in this case, though, without digging through a number of other tests and changing their util method calls, which is yet more diff in this PR :/
/// Gets the SCID which should be used to identify this channel for inbound payments. This | ||
/// should be used for providing invoice hints or in any other context where our counterparty | ||
/// will forward a payment to us. | ||
pub fn get_inbound_payment_scid(&self) -> Option<u64> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Consider dropping the get_
prefix here, too.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Personally I'm kinda a fan of the get_
prefix, they communicate the functions behavior (really, that it doesn't have side-effects) in a useful way. Its a bit less rust-idiomatic, but its a bit more java-idiomatic, and I dunno about swift/JS, so I'm not sure that's a strong reason.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess my counter would be naming something that doesn't start with a verb indicates that it is a property.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In some idioms yes, but in Java isn't get_
expected, even for a property-fetcher?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, agreed. That is idiomatic for Java as I understand. Suppose we could always tag these somehow for the bindings generator, but no need to worry about this now.
bee9478
to
98c8ca5
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks good to me, done several passes now. Happy to start looking onto to part 2!
`handle_monitor_err!()` has a number of different forms depending on which messages and actions were outstanding when the monitor updating first failed. Instead of matching by argument count, its much more readable to put an explicit string in the arguments to make it easy to scan for the called form.
New `funding_locked` messages can include SCID aliases which our counterparty will recognize as "ours" for the purposes of relaying transactions to us. This avoids telling the world about our on-chain transactions every time we want to receive a payment, and will allow for receiving payments before the funding transaction appears on-chain. Here we store the new SCID aliases and use them in invoices instead of he "standard" SCIDs.
This avoids needing to update channel closure code in many places as we add multiple SCIDs for each channel and have to track them.
This creates an SCID alias for all of our outbound channels, which we send to our counterparties as a part of the `funding_locked` message and then recognize in any HTLC forwarding instructions. Note that we generate an SCID alias for all channels, including already open ones, even though we currently have no way of communicating to our peers the SCID alias for already-open channels.
As a part of adding SCID aliases to channels, we now have to accept otherwise-redundant funding_locked messages which serve only to update the SCID alias. Previously, we'd failt he channel as such an update used to be bogus.
98c8ca5
to
e4486fe
Compare
Squashed the fixup commits down. |
pub fn create_unannounced_chan_between_nodes_with_value<'a, 'b, 'c, 'd>(nodes: &'a Vec<Node<'b, 'c, 'd>>, a: usize, b: usize, channel_value: u64, push_msat: u64, a_flags: InitFeatures, b_flags: InitFeatures) -> (msgs::FundingLocked, Transaction) { | ||
let mut no_announce_cfg = test_default_channel_config(); | ||
no_announce_cfg.channel_options.announced_channel = false; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No problem with passing on this. My concern was more with code repetition resulting in needing to modify two places rather than reducing lines of code.
/// Gets the SCID which should be used to identify this channel for inbound payments. This | ||
/// should be used for providing invoice hints or in any other context where our counterparty | ||
/// will forward a payment to us. | ||
pub fn get_inbound_payment_scid(&self) -> Option<u64> { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess my counter would be naming something that doesn't start with a verb indicates that it is a property.
Fuzz CI failure is an upstream regression, will fix separately. Kicked CI to hopefully make it pass here. |
Good reviewing again quickly rn and starting to look on PR 2. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ACK e4486fe
This is part 1/3 of the 0conf PR set, which only adds support for SCID aliaes using the new field in
funding_locked
. It does not yet support the relevant feature bit which is going to be a chunk more work as it will require fully implementing channel type negotiation, something we're currently missing (that will be PR 2). PR 3 will then add actual 0conf support, which isn't too bad after the first two land.This and all the 0conf PRs will need a very careful review. I'd especially recommend reviewers scan through all our existing uses of
short_channel_id
and familiarize yourself with them either for this or later 0conf PRs as most uses of them will need to change.Based on the 1.41.1 MSRV bump cause I'm lazy.