channeldb+cnct+invoices: track invoice htlcs #3390
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
joostjager
force-pushed
the
invoice-circuits
branch
3 times, most recently
from
323e1a4 to
3bc173c
Compare
joostjager
force-pushed
the
invoice-circuits
branch
4 times, most recently
from
1f2610f to
c3ada97
Compare
joostjager
force-pushed
the
invoice-circuits
branch
2 times, most recently
from
085d358 to
152f4a3
Compare
This was referenced Aug 16, 2019
joostjager
force-pushed
the
invoice-circuits
branch
from
152f4a3 to
c62061a
Compare
joostjager
force-pushed
the
invoice-circuits
branch
from
c62061a to
927a4a8
Compare
joostjager
force-pushed
the
invoice-circuits
branch
2 times, most recently
from
e005e22 to
1267758
Compare
joostjager
changed the title
wpaulino
added
database
Roasbeef
requested changes
Aug 20, 2019
joostjager
force-pushed
the
invoice-circuits
branch
from
1267758 to
db53cb1
Compare
joostjager
force-pushed
the
invoice-circuits
branch
from
e52afb7 to
d04561a
Compare
Contributor
Author
|
While working on mpp and with amp in mind, I already saw the extra invoice htlc fields coming. Added a commit |
joostjager
force-pushed
the
invoice-circuits
branch
3 times, most recently
from
0d6bd4d to
445d9ff
Compare
Contributor
Author
|
Also modified old migration calls to Migrations that aren't fully self-contained are dangerous. |
joostjager
force-pushed
the
invoice-circuits
branch
from
445d9ff to
b8d8389
Compare
cfromknecht
approved these changes
Sep 4, 2019
Contributor
cfromknecht
left a comment
cfromknecht
left a comment
There was a problem hiding this comment.
LGTM ✅ excited to start using tlv in our database, will make any future "migrations" to invoice htlcs straightforward :D
Contributor
|
(still a stray fixup commit btw) |
Previously a check was made for accepted and settled invoices against the paid amount. This opens up a probe vector where an attacker can pay to an invoice with an amt that is higher than the invoice amount and find out if the invoice is already paid or not.
Add logic to specifically exercise the replay behavior of invoice registry for hodl invoices.
Currently the invoice registry cannot tell apart the htlcs that pay to an invoice. Because htlcs may also be replayed on startup, it isn't possible to determine the total amount paid to an invoice. This commit is a first step towards fixing that. It reports the circuit keys of htlcs to the invoice registry, which forms the basis for accurate invoice accounting.
This commit adds a set of htlcs to the Invoice struct and serializes/deserializes this set to/from disk. It is a preparation for accurate invoice accounting across restarts of lnd. A migration is added for the invoice htlcs. In addition to these changes, separate final cltv delta and expiry invoice fields are created and populated. Previously it was required to decode this from the stored payment request. The reason to create a combined commit is to prevent multiple migrations.
Now that the Invoice struct contains the decoded final cltv delta value, the decoding of payment requests can be removed from the invoice registry.
As the logic around invoice mutations gets more complex, the friction caused by having this logic split between invoice registry and channeldb becomes more apparent. This commit brings a clearer separation of concerns by centralizing the accept/settle logic in the invoice registry. The original AcceptOrSettle method is renamed to UpdateInvoice because the update to perform is controlled by the callback.
This commit is a continuation of the centralization of invoice state transition logic in the invoice registry.
This commit is a continuation of the centralization of invoice state transition logic in the invoice registry.
This commit refactors the invoice registry accept/settle logic so that it doesn't rely anymore on a set of error values to indirectly communicate from the update callback to the main function what action is required on the htlc.
Needed for time control in unit tests.
Previously the invoice registry wasn't aware of replayed htlcs. This was dealt with by keeping the invoice accept/settle logic idempotent, so that a replay wouldn't have an effect. This mechanism has two limitations: 1. No accurate tracking of the total amount paid to an invoice. The total amount couldn't just be increased with every htlc received, because it could be a replay which would lead to counting the htlc amount multiple times. Therefore the total amount was set to the amount of the first htlc that was received, even though there may have been multiple htlcs paying to the invoice. 2. Impossible to check htlc expiry consistently for hodl invoices. When an htlc is new, its expiry needs to be checked against the invoice cltv delta. But for a replay, that check must be skipped. The htlc was accepted in time, the invoice was moved to the accepted state and a replay some blocks later shouldn't lead to that htlc being cancelled. Because the invoice registry couldn't recognize replays, it stopped checking htlc expiry heights when the invoice reached the accepted state. This prevents hold htlcs from being cancelled after a restart. But unfortunately this also caused additional htlcs to be accepted on an already accepted invoice without their expiry being checked. In this commit, the invoice registry starts to persistently track htlcs so that replays can be recognized. For replays, an htlc resolution action is returned early. This fixes both limitations mentioned above.
joostjager
force-pushed
the
invoice-circuits
branch
from
b8d8389 to
45cd76e
Compare
Contributor
Author
|
fixup squashed |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR modifies invoice registry so that it tracks the individual htlcs paying to an invoice. This leads to the following improvements:
Report the amount paid to an invoice correctly. Previously only the amount of the first HTLC was reported. With this PR, that is changed to the sum of all settled HTLCs.
Report channel through which an invoice was paid.
Consistent HTLC acceptance for hodl invoices. Previously the HTLC expiry check was skipped in certain situations which could lead to channel closure.
More generally, this PR adds the fine-grained HTLC control that is required for multi path payments.
NOTE: The migration of invoices in this pr cannot fill in the invoice htlcs for existing invoices. If there are invoices in the
acceptedstate during the upgrade, the held htlcs may be released on startup. This risk can be eliminated by ensuring there are no invoices in theacceptedstate.Fixes #3169
Fixes #2208