Avoid cascading failure: give up on incoming HTLCs in time if outgoing is stuck. #6378
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rustyrussell
added
the
protocol
rustyrussell
force-pushed
the
guilt/give-up-onb-htlcs
branch
from
13a190f
to
0103299
Compare
I shut down bitcoind during a test, and bcli leak reports flooded in. They're all temporary, but this fixes them. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
Caught by leak detection, we just re-assigned this when we retried: sure, it's temporary, but it's technically a leak. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
rustyrussell
force-pushed
the
guilt/give-up-onb-htlcs
branch
from
0103299
to
91aca34
Compare
|
Trivial rebase. |
rustyrussell
force-pushed
the
guilt/give-up-onb-htlcs
branch
from
91aca34
to
bb3a4b3
Compare
…losing on us. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
The test actually triggers this: 1. We don't get our commitment tx mined at all (we block it). 2. By the time the peer does, the HTLC is expired. 3. We have the preimage but we don't even try, since it's expired. We should at least *try* to collect the HTLC in this case. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au>
This cause of cascading failure was pointed out by @t-bast: if fees spike and you don't timeout an outgoing onchain HTLC, you should nonetheless fail the incoming htlc because otherwise the incoming peer will close on you. Of course, there's a risk of losing funds, but this only happens if you weren't going to get the HTLC spend in time anyway. And it would also catch any other reason that the downstream onchain goes wrong, containing the damage. Signed-off-by: Rusty Russell <rusty@rustcorp.com.au> Reported-by: @t-bast Changelog-Fixed: Protocol: We will close incoming HTLCs early if the outgoing HTLC is stuck onchain long enough, to avoid cascating failure.
rustyrussell
force-pushed
the
guilt/give-up-onb-htlcs
branch
from
bb3a4b3
to
fd4b778
Compare
Comment on lines
+1506
to
+1511
| if (outs[i]->resolved->tx_type != SELF) { | ||
| status_broken("HTLC already resolved by %s" | ||
| " when we found preimage", | ||
| tx_type_name(outs[i]->resolved->tx_type)); | ||
| return; | ||
| } |
There was a problem hiding this comment.
we can remove the {..} due the single stmt inside the if?
Comment on lines
+1761
to
+1764
| && !hout->in->preimage) { | ||
| local_fail_in_htlc(hout->in, | ||
| take(towire_permanent_channel_failure(NULL))); | ||
| } |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
@t-bast described this issue in general, and here's the fix, with some cleanups on the way.
Our solution is to monitor for this:
This is a risk, of course: the outgoing HTLC could be claimed by the peer. But that's no worse to us than it not getting mined at all, which we were prepared for.