cnct+htlcswitch+invoices: move invoice parameter check to registry by joostjager · Pull Request #2913 · lightningnetwork/lnd

joostjager · 2019-04-08T14:35:00Z

This PR simplifies the link implementation by moving out all references to invoices. As an exit hop, the link offers its htlc to the invoice registry. The invoice registry decides how and when the link should respond to the htlc.

Checking of the cltv delta and amount is now performed in the invoice registry. This concentrates the logic around invoice acceptance and settling in the invoice registry. It increases maintainability and is a also preparatory step for implementation of multi path payments.

In addition to that, moving the htlc parameter check to the invoice registry allows us to apply the exact same logic for on-chain resolution of an incoming htlc. This fixes some unaddressed edge cases:

On-chain resolution of an incoming htlc for a canceled invoice. Previously the htlc would still be settled even though the invoice database indicates that we should reject payments to the invoice. The htlc was pulled but the invoice stayed in the canceled stayed.
On-chain resolution of an incoming htlc for an open invoice. Previously the invoice was marked settled as long as the htlc hadn't expired yet. But if the number of remaining blocks until expiry is low, it could
happen that we couldn't pull the htlc in the end.
On-chain resolution of an incoming htlc for a hodl invoice. Previously the hodl invoice would move to the accepted state regardless of the number of remaining blocks until htlc expiry. It could be that invoices move to the accepted state without meeting the invoice min cltv delta requirement. This violates the assumption of the user that there are still enough blocks left when the accepted state transition is received.

To reduce the number of execution paths for incoming htlc resolution, this PR modifies the channel arbitrator to always send htlcs via the incoming contest resolver. This allows us to perform the check with the invoice registry in a single location.

Furthermore this change makes it unnecessary to already determine earlier in the process whether an incoming contest or a success resolver is needed. This simplifies channel and channel_arbitrator.

joostjager · 2019-04-11T20:07:06Z

Linking #2935. This may make it easier to handle hodl invoices and incoming preimages in the contract resolvers.

cfromknecht

great work @joostjager, completed an initial round of review

contractcourt/interfaces.go

server.go

lnwallet/channel.go

contractcourt/htlc_incoming_contest_resolver.go

htlcswitch/link.go

cfromknecht · 2019-05-07T02:00:32Z

htlcswitch/link.go

not sure that converting to int32 helps here. it's still susceptible to underflow, which would later satisfy either of the equations: blocksTilExpiry >= finalExpiry or blocksTilExpiry >= finalRejectDelta. however, checking that pd.Timeout > heightNow only needs to be on the first accept, which goes in line with my other comment passing both distinctly into NotifyExitHopHtlc and validating this in the invoice registry

Suppose pd.Timeout is 3 and heightNow is 5. blocksToExpiry would then be -2 and blocksTilExpiry >= finalExpiry = -2 >= finalExpiry would be false?

But this check has now indeed been moved to invoice registry.

The case I'm thinking about is when pd.Timeout is very high (as a unit32), like 0x80000000. In this example, blocksTilExpiry is now 2147483643 which exceeds any reasonable finalExpriy

Rewritten the comparison without subtraction

cfromknecht · 2019-05-07T10:17:18Z

contractcourt/htlc_incoming_contest_resolver.go

are we sure we want to do this? seems like if the htlc is unswept by either party the channel should remain unclosed. perhaps there's an argument to be made that in this case we should offload to the stray output pool, to decouple it from the closing of the channel. but that seems easier to add on later if the contract hasn't been closed, than trying to resurrect everything after the resolver state has been cleaned up

It was already like that, I just clarified the rationale in the comment. Would you like to change that behaviour in this PR? I am not sure how users will react to these stuck channels, as long as we don't have a stray output pool.

Btw, for exit hops this check is actually unnecessary, because further down NotifyExitHopHtlc will also return a cancel action. For exit hops we really want to close the contract, because accepting it would violate the invoice final cltv requirement?

@joostjager doesn't need to be done in this pr, that is a bigger change and currently we are keeping the existing behavior.

I am not sure how users will react to these stuck channels

i'm not sure either, though imo it'd be preferable to know that we are possibly leaving money on the table, especially since the preimage could come in shortly after the expiry. it's also possible that this whole flow can be delayed due to fee spikes, and in that case it seems like it'd be premature to give up on the HTLC, though i admit i haven't worked through all the edge cases

accepting it would violate the invoice final cltv requirement

i'm not positive what the correct behavior is either. in the case of a regular invoice, we will already know the preimage and the resolver will convert itself to a success resolver and we no longer have this issue. if we have a hold invoice, "learning" the preimage after the expiry could only be done by the preimage being provided externally via rpc, so perhaps that constitutes some explicit consent that the user in fact wants to settle the invoice even if the expiry has expired. what do you think?

this whole flow can be delayed due to fee spikes, and in that case it seems like it'd be premature to give up on the HTLC

Maybe, but in the other hand, no one is losing money. The htlc is canceled back and we never marked it as settled, so no pizza has been delivered yet.

if we have a hold invoice, "learning" the preimage after the expiry could only be done by the preimage being provided externally via rpc, so perhaps that constitutes some explicit consent that the user in fact wants to settle the invoice even if the expiry has expired.

It is about accepting the invoice. For hodl invoices, the preimage isn't needed to accept. If we would accept, the user should check the height when the accepted event comes because there may be fewer blocks left than expected.

Also for regular invoices, there is a risk in marking the invoice settled when there are not enough blocks left and we still need to wait for confirmation of a tx spending the commit output. We normally assume that we need the final cltv delta blocks to get those confirmed. This is also the criterium for settling invoices in the link.

Previously we would only mark the invoice as settled when the success resolver completes. This is safer, but can also be an unnecessary delay in case there are plenty of blocks left. It will take longer for the pizza to be delivered because of a channel failure that the payer has nothing to do with.

lnwallet/channel.go

contractcourt/channel_arbitrator.go

lnd_multihoplocalchainclaim_test.go

Roasbeef · 2019-05-11T01:17:10Z

channeldb/invoice_test.go

I think this commit could've easily been split up into multiple commits. There's a lot going on in this commit across multiple files and packages.

How would you do this easily? This commit is a set of changes that are all interconnected. Leaving some out will not result in a working lnd. I tried to split off all independent changes already. Also, I want to fix tests in the same commit that breaks them, so those can't be moved out.

Yeah there's a tradeoff w.r.t having them always compile vs having them make smaller changes at a time with an eye for ease of review. IMO things are more likely to slip by with larger commits that make modifications over several packages/files. Isolated commits also allow the committer to tell a narrative in a sense, with the progression of each commit.

As it's already received some attention as is, I think it's too late at this point. However, as an example, the AcceptOrSettleInvoice change could've been added in a distinct commit that just left all the logic in place, but used a nil value for the function closure in the link. A follow up commit could then remove the old verification logic and add the height based verification to that new closure.

Ok, that is a good example indeed. Could have been a separate commit. Not sure about committing non-building code though.

invoices/invoiceregistry.go

lnd_multihopreceiverchainclaim_test.go

joostjager · 2019-05-13T09:57:50Z

Then the commitment tx is published and mined, after which the resolver takes over. At that point there are just 39 blocks left. According to the new (consistent) logic introduced in this PR, that is not enough and the htlc is canceled.

Why should the HTLC be cancelled? It should be swept since Carol has the preimage.

See explanation in comments above. Having the preimage for an invoice doesn't always mean we should settle the htlc. At least that is the idea introduced in this PR.

joostjager · 2019-05-13T10:10:09Z

@Roasbeef @cfromknecht ptal

Roasbeef

LGTM on the code changes 🐣

Starting to run this on some testnet nodes, also thinking about how to properly evaluate it in the wild before rolling out. It's mostly code movement, but existing code pathways are also modified.

cfromknecht

@joostjager did another pass, looking good! a couple non-blocking comments, otherwise LGTM 🌪

One step closer to multi-path payments!

cfromknecht · 2019-05-15T02:26:13Z

lnd_multi-hop_htlc_local_chain_claim_test.go

imo it'd be preferable to start moving these all into the lntest package, tho i suspect there are a bunch of dependencies either in lnd or just helper functions that would break if we don't do the move all at once. is that the case?

It needs to be done all at once, commit added.

cfromknecht · 2019-05-15T02:30:52Z

lnd_multi-hop_htlc_remote_chain_claim_test.go

this block might make a good helper function, since it's used 3+ times

joostjager · 2019-05-15T07:49:35Z

@cfromknecht ptal

One of the first things the incoming contest resolver does is checking if the preimage is available and if it is, convert itself into a success resolver. This behaviour makes it unnecessary to already determine earlier in the process whether an incoming contest or a success resolver is needed. By having all incoming htlcs go through the incoming contest resolver, the number of execution paths is reduced and it becomes easier to ascertain that the implemented logic is correct. The only functional change in this commit is that a forwarded htlc for which is the preimage is known, is no longer settled when the htlc is already expired. Previously a success resolver would be instantiated directly, skipping the expiry height check. This created a risk that the success resolver would never finish, because an expired htlc could already have been swept by the remote party and there is no detection of this remote spend in the success resolver currently. With the new change, the general direction that an expired htlc shouldn't be settled and instead given up on is implemented more consistently. This commit prepares for fixing edges cases related to hodl invoice on-chain resolution.

Now that the success resolver preimage field is always populated by the incoming contest resolver, preimage lookups earlier in the process (channel and channel arbitrator) can mostly be removed.

New behaviour of the chain notifier to always send the current block immediately after registration takes away the need to make a separate GetBestBlock call on ChainIO.

The former tryApplyPreimage function silently ignored invalid preimages. This could mask potential bugs. This commit makes the logic stricter and generates an error in case an unexpected mismatch occurs.

This commit isolates preimages of forwarded htlcs from invoice preimages. The reason to do this is to prevent the incoming contest resolver from settling exit hop htlcs for which the invoice isn't marked as settled.

Create an interface type to be able to mock the registry in unit tests.

This commit splits out several integration tests that will be modified in a follow up commit to separate files. The current lnd_test.go file is 10k+ loc which makes it harder to handle by tools and it isn't good for overview either.

This commit is the final step in making the link unaware of invoices. It now purely offers the htlc to the invoice registry and follows instructions from the invoice registry about how and when to respond to the htlc. The change also fixes a bug where upon restart, hodl htlcs were subjected to the invoice minimum cltv delta requirement again. If the block height has increased in the mean while, the htlc would be canceled back. Furthermore the invoice registry interaction is aligned between link and contract resolvers.

This commit refactors test code around channel restoration in unit tests to make it easier to use.

This commit adds a test that covers the hodl invoice behaviour after a link restart.

joostjager mentioned this pull request Apr 8, 2019

htlcswitch: do not check final cltv for accepted invoices #2915

Merged

joostjager force-pushed the hodl-restart-fix branch 3 times, most recently from 4b59b01 to aaa41e7 Compare April 9, 2019 12:59

joostjager changed the title ~~Hodl restart fix~~ cnct+htlcswitch+invoices: move invoice parameter to check registry Apr 9, 2019

joostjager changed the title ~~cnct+htlcswitch+invoices: move invoice parameter to check registry~~ cnct+htlcswitch+invoices: move invoice parameter check to registry Apr 9, 2019

joostjager force-pushed the hodl-restart-fix branch from aaa41e7 to 5358fa0 Compare April 9, 2019 13:17

joostjager marked this pull request as ready for review April 9, 2019 15:12

joostjager changed the title ~~cnct+htlcswitch+invoices: move invoice parameter check to registry~~ cnct+htlcswitch+invoices: move invoice parameter check to registry [wip] Apr 9, 2019

cfromknecht added this to the 0.7 milestone Apr 11, 2019

joostjager force-pushed the hodl-restart-fix branch 2 times, most recently from a3d673d to 6a9924e Compare April 17, 2019 09:27

joostjager mentioned this pull request Apr 17, 2019

cnct: fix on-chain hodl invoice handling and simplify incoming resolver flow #2955

Closed

joostjager force-pushed the hodl-restart-fix branch 5 times, most recently from ff5227b to 89a8545 Compare April 17, 2019 11:31

Roasbeef added htlcswitch payments Related to invoices/payments refactoring safety General label for issues/PRs related to the safety of using the software labels Apr 20, 2019

Roasbeef requested a review from cfromknecht April 26, 2019 00:47

joostjager mentioned this pull request Apr 26, 2019

Payment was received, but settle is false #2909

Closed

joostjager force-pushed the hodl-restart-fix branch from 89a8545 to 6d58b4e Compare May 6, 2019 20:01

joostjager changed the title ~~cnct+htlcswitch+invoices: move invoice parameter check to registry [wip]~~ cnct+htlcswitch+invoices: move invoice parameter check to registry May 6, 2019

cfromknecht reviewed May 7, 2019

View reviewed changes

joostjager force-pushed the hodl-restart-fix branch from 6d58b4e to dc54add Compare May 7, 2019 13:25

Roasbeef requested changes May 11, 2019

View reviewed changes

joostjager force-pushed the hodl-restart-fix branch from fe71137 to a3d22c4 Compare May 13, 2019 09:58

Roasbeef approved these changes May 14, 2019

View reviewed changes

cfromknecht approved these changes May 15, 2019

View reviewed changes

joostjager force-pushed the hodl-restart-fix branch 4 times, most recently from 49f76db to 154f688 Compare May 15, 2019 07:43

joostjager requested a review from cfromknecht May 15, 2019 12:32

joostjager and others added 14 commits May 15, 2019 14:41

channel+cnct: remove preimage from channel and resolution

d55a8b7

Now that the success resolver preimage field is always populated by the incoming contest resolver, preimage lookups earlier in the process (channel and channel arbitrator) can mostly be removed.

cnct: do not depend on ChainIO in incoming contest resolver

ec6a35d

New behaviour of the chain notifier to always send the current block immediately after registration takes away the need to make a separate GetBestBlock call on ChainIO.

cnct: be stricter about matching preimages

99e42dd

The former tryApplyPreimage function silently ignored invalid preimages. This could mask potential bugs. This commit makes the logic stricter and generates an error in case an unexpected mismatch occurs.

witness_beacon: do not look up invoice preimages

1a80a1e

This commit isolates preimages of forwarded htlcs from invoice preimages. The reason to do this is to prevent the incoming contest resolver from settling exit hop htlcs for which the invoice isn't marked as settled.

cnct: add invoice registry interface

d8dac49

Create an interface type to be able to mock the registry in unit tests.

lntest/node: set up InvoicesClient for HarnessNode

b917820

lnd_test: move chain claim tests to separate files

bf0af12

This commit splits out several integration tests that will be modified in a follow up commit to separate files. The current lnd_test.go file is 10k+ loc which makes it harder to handle by tools and it isn't good for overview either.

lnd_test: add hodl parameter to createThreeHopNetwork

e095819

cnct/test: extend mockWitnessBeacon

16ff4e3

cnct/test: add incoming contest resolver test

3d17c2b

htlcswitch/test: use single channel restore function

e5ead59

This commit refactors test code around channel restoration in unit tests to make it easier to use.

htlcswitch/test: hodl invoice restart test

570f9ca

This commit adds a test that covers the hodl invoice behaviour after a link restart.

joostjager force-pushed the hodl-restart-fix branch 2 times, most recently from cdc834f to 570f9ca Compare May 16, 2019 18:21

joostjager merged commit 27ae22f into lightningnetwork:master May 16, 2019

joostjager deleted the hodl-restart-fix branch May 16, 2019 19:31

Conversation

joostjager commented Apr 8, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

joostjager commented Apr 11, 2019

Uh oh!

cfromknecht left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

cfromknecht May 7, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joostjager May 7, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

joostjager commented May 13, 2019

Uh oh!

joostjager commented May 13, 2019

Uh oh!

Roasbeef left a comment

Choose a reason for hiding this comment

Uh oh!

cfromknecht left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

joostjager commented May 15, 2019

Uh oh!

Reviewers

Assignees

joostjager commented Apr 8, 2019 •

edited

Loading

cfromknecht left a comment •

edited

Loading

cfromknecht May 7, 2019 •

edited

Loading

joostjager May 7, 2019 •

edited

Loading