Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cnct+htlcswitch+invoices: move invoice parameter check to registry #2913

Merged
merged 14 commits into from May 16, 2019

Conversation

Projects
None yet
4 participants
@joostjager
Copy link
Collaborator

commented Apr 8, 2019

This PR simplifies the link implementation by moving out all references to invoices. As an exit hop, the link offers its htlc to the invoice registry. The invoice registry decides how and when the link should respond to the htlc.

Checking of the cltv delta and amount is now performed in the invoice registry. This concentrates the logic around invoice acceptance and settling in the invoice registry. It increases maintainability and is a also preparatory step for implementation of multi path payments.

In addition to that, moving the htlc parameter check to the invoice registry allows us to apply the exact same logic for on-chain resolution of an incoming htlc. This fixes some unaddressed edge cases:

  • On-chain resolution of an incoming htlc for a canceled invoice. Previously the htlc would still be settled even though the invoice database indicates that we should reject payments to the invoice. The htlc was pulled but the invoice stayed in the canceled stayed.

  • On-chain resolution of an incoming htlc for an open invoice. Previously the invoice was marked settled as long as the htlc hadn't expired yet. But if the number of remaining blocks until expiry is low, it could
    happen that we couldn't pull the htlc in the end.

  • On-chain resolution of an incoming htlc for a hodl invoice. Previously the hodl invoice would move to the accepted state regardless of the number of remaining blocks until htlc expiry. It could be that invoices move to the accepted state without meeting the invoice min cltv delta requirement. This violates the assumption of the user that there are still enough blocks left when the accepted state transition is received.

To reduce the number of execution paths for incoming htlc resolution, this PR modifies the channel arbitrator to always send htlcs via the incoming contest resolver. This allows us to perform the check with the invoice registry in a single location.

Furthermore this change makes it unnecessary to already determine earlier in the process whether an incoming contest or a success resolver is needed. This simplifies channel and channel_arbitrator.

@joostjager joostjager force-pushed the joostjager:hodl-restart-fix branch 3 times, most recently from 4b59b01 to aaa41e7 Apr 9, 2019

@joostjager joostjager changed the title Hodl restart fix cnct+htlcswitch+invoices: move invoice parameter to check registry Apr 9, 2019

@joostjager joostjager changed the title cnct+htlcswitch+invoices: move invoice parameter to check registry cnct+htlcswitch+invoices: move invoice parameter check to registry Apr 9, 2019

@joostjager joostjager force-pushed the joostjager:hodl-restart-fix branch from aaa41e7 to 5358fa0 Apr 9, 2019

@joostjager joostjager marked this pull request as ready for review Apr 9, 2019

@joostjager joostjager changed the title cnct+htlcswitch+invoices: move invoice parameter check to registry cnct+htlcswitch+invoices: move invoice parameter check to registry [wip] Apr 9, 2019

@cfromknecht cfromknecht added this to the 0.7 milestone Apr 11, 2019

@joostjager

This comment has been minimized.

Copy link
Collaborator Author

commented Apr 11, 2019

Linking #2935. This may make it easier to handle hodl invoices and incoming preimages in the contract resolvers.

@joostjager joostjager force-pushed the joostjager:hodl-restart-fix branch 2 times, most recently from a3d673d to 6a9924e Apr 17, 2019

@joostjager joostjager force-pushed the joostjager:hodl-restart-fix branch 5 times, most recently from ff5227b to 89a8545 Apr 17, 2019

@Roasbeef Roasbeef requested a review from cfromknecht Apr 26, 2019

@joostjager joostjager force-pushed the joostjager:hodl-restart-fix branch from 89a8545 to 6d58b4e May 6, 2019

@joostjager joostjager changed the title cnct+htlcswitch+invoices: move invoice parameter check to registry [wip] cnct+htlcswitch+invoices: move invoice parameter check to registry May 6, 2019

@cfromknecht
Copy link
Collaborator

left a comment

great work @joostjager, completed an initial round of review

Show resolved Hide resolved contractcourt/interfaces.go
Show resolved Hide resolved server.go Outdated
Show resolved Hide resolved lnwallet/channel.go
Show resolved Hide resolved contractcourt/htlc_incoming_contest_resolver.go
Show resolved Hide resolved contractcourt/htlc_incoming_contest_resolver.go
Show resolved Hide resolved htlcswitch/link.go
Show resolved Hide resolved htlcswitch/link.go Outdated
Show resolved Hide resolved htlcswitch/link.go
Show resolved Hide resolved htlcswitch/link.go Outdated
@@ -2868,8 +2782,12 @@ func (l *channelLink) processExitHop(pd *lnwallet.PaymentDescriptor,
// Notify the invoiceRegistry of the exit hop htlc. If we crash right
// after this, this code will be re-executed after restart. We will
// receive back a resolution event.
blocksToExpiry := int32(pd.Timeout) - int32(heightNow)

This comment has been minimized.

Copy link
@cfromknecht

cfromknecht May 7, 2019

Collaborator

not sure that converting to int32 helps here. it's still susceptible to underflow, which would later satisfy either of the equations: blocksTilExpiry >= finalExpiry or blocksTilExpiry >= finalRejectDelta. however, checking that pd.Timeout > heightNow only needs to be on the first accept, which goes in line with my other comment passing both distinctly into NotifyExitHopHtlc and validating this in the invoice registry

This comment has been minimized.

Copy link
@joostjager

joostjager May 7, 2019

Author Collaborator

Suppose pd.Timeout is 3 and heightNow is 5. blocksToExpiry would then be -2 and blocksTilExpiry >= finalExpiry = -2 >= finalExpiry would be false?

This comment has been minimized.

Copy link
@joostjager

joostjager May 7, 2019

Author Collaborator

But this check has now indeed been moved to invoice registry.

This comment has been minimized.

Copy link
@cfromknecht

cfromknecht May 8, 2019

Collaborator

The case I'm thinking about is when pd.Timeout is very high (as a unit32), like 0x80000000. In this example, blocksTilExpiry is now 2147483643 which exceeds any reasonable finalExpriy

This comment has been minimized.

Copy link
@joostjager

joostjager May 8, 2019

Author Collaborator

Rewritten the comparison without subtraction

// sender can sweep it, so we'll end our lifetime. Here we deliberately
// forego the chance that the sender doesn't sweep and we already have
// or will learn the preimage. Otherwise the resolver could potentially
// stay active indefinitely and the channel will never close properly.

This comment has been minimized.

Copy link
@cfromknecht

cfromknecht May 7, 2019

Collaborator

are we sure we want to do this? seems like if the htlc is unswept by either party the channel should remain unclosed. perhaps there's an argument to be made that in this case we should offload to the stray output pool, to decouple it from the closing of the channel. but that seems easier to add on later if the contract hasn't been closed, than trying to resurrect everything after the resolver state has been cleaned up

This comment has been minimized.

Copy link
@joostjager

joostjager May 7, 2019

Author Collaborator

It was already like that, I just clarified the rationale in the comment. Would you like to change that behaviour in this PR? I am not sure how users will react to these stuck channels, as long as we don't have a stray output pool.

Btw, for exit hops this check is actually unnecessary, because further down NotifyExitHopHtlc will also return a cancel action. For exit hops we really want to close the contract, because accepting it would violate the invoice final cltv requirement?

This comment has been minimized.

Copy link
@cfromknecht

cfromknecht May 15, 2019

Collaborator

@joostjager doesn't need to be done in this pr, that is a bigger change and currently we are keeping the existing behavior.

I am not sure how users will react to these stuck channels

i'm not sure either, though imo it'd be preferable to know that we are possibly leaving money on the table, especially since the preimage could come in shortly after the expiry. it's also possible that this whole flow can be delayed due to fee spikes, and in that case it seems like it'd be premature to give up on the HTLC, though i admit i haven't worked through all the edge cases

accepting it would violate the invoice final cltv requirement

i'm not positive what the correct behavior is either. in the case of a regular invoice, we will already know the preimage and the resolver will convert itself to a success resolver and we no longer have this issue. if we have a hold invoice, "learning" the preimage after the expiry could only be done by the preimage being provided externally via rpc, so perhaps that constitutes some explicit consent that the user in fact wants to settle the invoice even if the expiry has expired. what do you think?

This comment has been minimized.

Copy link
@joostjager

joostjager May 15, 2019

Author Collaborator

this whole flow can be delayed due to fee spikes, and in that case it seems like it'd be premature to give up on the HTLC

Maybe, but in the other hand, no one is losing money. The htlc is canceled back and we never marked it as settled, so no pizza has been delivered yet.

if we have a hold invoice, "learning" the preimage after the expiry could only be done by the preimage being provided externally via rpc, so perhaps that constitutes some explicit consent that the user in fact wants to settle the invoice even if the expiry has expired.

It is about accepting the invoice. For hodl invoices, the preimage isn't needed to accept. If we would accept, the user should check the height when the accepted event comes because there may be fewer blocks left than expected.

Also for regular invoices, there is a risk in marking the invoice settled when there are not enough blocks left and we still need to wait for confirmation of a tx spending the commit output. We normally assume that we need the final cltv delta blocks to get those confirmed. This is also the criterium for settling invoices in the link.

Previously we would only mark the invoice as settled when the success resolver completes. This is safer, but can also be an unnecessary delay in case there are plenty of blocks left. It will take longer for the pizza to be delivered because of a channel failure that the payer has nothing to do with.

@joostjager joostjager force-pushed the joostjager:hodl-restart-fix branch from 6d58b4e to dc54add May 7, 2019

@joostjager

This comment has been minimized.

Copy link
Collaborator Author

commented May 13, 2019

Then the commitment tx is published and mined, after which the resolver takes over. At that point there are just 39 blocks left. According to the new (consistent) logic introduced in this PR, that is not enough and the htlc is canceled.

Why should the HTLC be cancelled? It should be swept since Carol has the preimage.

See explanation in comments above. Having the preimage for an invoice doesn't always mean we should settle the htlc. At least that is the idea introduced in this PR.

@joostjager joostjager force-pushed the joostjager:hodl-restart-fix branch from fe71137 to a3d22c4 May 13, 2019

@joostjager

This comment has been minimized.

Copy link
Collaborator Author

commented May 13, 2019

@Roasbeef
Copy link
Member

left a comment

LGTM on the code changes 🐣

Starting to run this on some testnet nodes, also thinking about how to properly evaluate it in the wild before rolling out. It's mostly code movement, but existing code pathways are also modified.

@cfromknecht
Copy link
Collaborator

left a comment

@joostjager did another pass, looking good! a couple non-blocking comments, otherwise LGTM 🌪

One step closer to multi-path payments!

@@ -0,0 +1,352 @@
// +build rpctest

This comment has been minimized.

Copy link
@cfromknecht

cfromknecht May 15, 2019

Collaborator

imo it'd be preferable to start moving these all into the lntest package, tho i suspect there are a bunch of dependencies either in lnd or just helper functions that would break if we don't do the move all at once. is that the case?

This comment has been minimized.

Copy link
@joostjager

joostjager May 15, 2019

Author Collaborator

It needs to be done all at once, commit added.

// Wait for carol to mark invoice as accepted. There is a small gap to
// bridge between adding the htlc to the channel and executing the exit
// hop logic.
for {

This comment has been minimized.

Copy link
@cfromknecht

cfromknecht May 15, 2019

Collaborator

this block might make a good helper function, since it's used 3+ times

This comment has been minimized.

Copy link
@joostjager

joostjager May 15, 2019

Author Collaborator

extracted

@joostjager joostjager force-pushed the joostjager:hodl-restart-fix branch 4 times, most recently from 49f76db to 154f688 May 15, 2019

@joostjager

This comment has been minimized.

Copy link
Collaborator Author

commented May 15, 2019

@joostjager joostjager requested a review from cfromknecht May 15, 2019

joostjager and others added some commits Apr 15, 2019

cnct: always create incoming contest resolver
One of the first things the incoming contest resolver does is checking
if the preimage is available and if it is, convert itself into a success
resolver.

This behaviour makes it unnecessary to already determine earlier in the
process whether an incoming contest or a success resolver is needed.

By having all incoming htlcs go through the incoming contest resolver,
the number of execution paths is reduced and it becomes easier to
ascertain that the implemented logic is correct.

The only functional change in this commit is that a forwarded htlc for
which is the preimage is known, is no longer settled when the htlc is
already expired. Previously a success resolver would be instantiated
directly, skipping the expiry height check.

This created a risk that the success resolver would never finish,
because an expired htlc could already have been swept by the remote
party and there is no detection of this remote spend in the success
resolver currently.

With the new change, the general direction that an expired htlc
shouldn't be settled and instead given up on is implemented more
consistently.

This commit prepares for fixing edges cases related to hodl
invoice on-chain resolution.
channel+cnct: remove preimage from channel and resolution
Now that the success resolver preimage field is always populated by the
incoming contest resolver, preimage lookups earlier in the
process (channel and channel arbitrator) can mostly be removed.
cnct: do not depend on ChainIO in incoming contest resolver
New behaviour of the chain notifier to always send the current block
immediately after registration takes away the need to make a separate
GetBestBlock call on ChainIO.
cnct: be stricter about matching preimages
The former tryApplyPreimage function silently ignored invalid preimages.
This could mask potential bugs. This commit makes the logic stricter and
generates an error in case an unexpected mismatch occurs.
witness_beacon: do not look up invoice preimages
This commit isolates preimages of forwarded htlcs from invoice
preimages. The reason to do this is to prevent the incoming contest
resolver from settling exit hop htlcs for which the invoice isn't marked
as settled.
cnct: add invoice registry interface
Create an interface type to be able to mock the registry in unit tests.
lnd_test: move chain claim tests to separate files
This commit splits out several integration tests that will be modified
in a follow up commit to separate files. The current lnd_test.go file is
10k+ loc which makes it harder to handle by tools and it isn't good for
overview either.
cnct+htlcswitch+invoices: move invoice parameter check out of link
This commit is the final step in making the link unaware of invoices. It
now purely offers the htlc to the invoice registry and follows
instructions from the invoice registry about how and when to respond to
the htlc.

The change also fixes a bug where upon restart, hodl htlcs were
subjected to the invoice minimum cltv delta requirement again. If the
block height has increased in the mean while, the htlc would be canceled
back.

Furthermore the invoice registry interaction is aligned between link and
contract resolvers.
htlcswitch/test: use single channel restore function
This commit refactors test code around channel restoration in unit
tests to make it easier to use.
htlcswitch/test: hodl invoice restart test
This commit adds a test that covers the hodl invoice behaviour after a
link restart.

@joostjager joostjager force-pushed the joostjager:hodl-restart-fix branch 2 times, most recently from cdc834f to 570f9ca May 15, 2019

@joostjager joostjager merged commit 27ae22f into lightningnetwork:master May 16, 2019

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
coverage/coveralls Coverage increased (+0.1%) to 60.341%
Details

@joostjager joostjager deleted the joostjager:hodl-restart-fix branch May 16, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.