Skip to content

channeldb+invoices: fix what pending invoice means in ChannelDB#3838

Merged
joostjager merged 2 commits intolightningnetwork:masterfrom
bhandras:i1404
Dec 19, 2019
Merged

channeldb+invoices: fix what pending invoice means in ChannelDB#3838
joostjager merged 2 commits intolightningnetwork:masterfrom
bhandras:i1404

Conversation

@bhandras
Copy link
Collaborator

@bhandras bhandras commented Dec 14, 2019

This pull request is a partial fix for issue #1404 where part of the problem comes from
not being able to filter out expired invoices due to those not being canceled
(fixed in PR #3694) while also having a misleading definition of what pending invoice
state means.

The PR changes the meaning of the pending_only flag in listinvoices
RPC to list invoices that are either in ContractOpen or ContractAccepted
state. Previously when the pending_only flag was set the listinvoices
RPC returned all invoices that are not settled which equals to invoices in ContractOpen, ContractAccepted or ContractCanceled state.

@bhandras bhandras changed the title channeldb+invoices: fix how isPending works in FetachAllInvoices. channeldb+invoices: fix what pending invoice means in ChannelDB Dec 16, 2019
Copy link
Contributor

@joostjager joostjager left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Straight-forward change. But it is a breaking rpc change. How to deal with that?

@bhandras
Copy link
Collaborator Author

bhandras commented Dec 16, 2019

Straight-forward change. But it is a breaking rpc change. How to deal with that?

To be fair #3694 already did this but this PR completes it.

@joostjager
Copy link
Contributor

To be fair #3694 already did this but this PR completes it.

How did it do that? It auto-canceled invoices, but it didn't change which invoices where returned when listing?

Copy link
Collaborator Author

@bhandras bhandras left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How did it do that? It auto-canceled invoices, but it didn't change which invoices where returned when listing?

You're correct, this change does indeed modify how pending_only flag works.
Looking at old behavior, pending meant: one of {accepted, open, canceled}; now pending means: one of {accepted, open}. When listing all invoices, behavior remains the same. With previous behavior users who were curious about actual pending (to be paid) invoices had to filter by hand. We can keep compatibility by adding a new flag, but imo it's more confusing.

@bhandras bhandras requested a review from joostjager December 16, 2019 12:12
@joostjager
Copy link
Contributor

We can keep compatibility by adding a new flag, but imo it's more confusing.

That is what we generally do. And mark the old flag as deprecated in at least one major release.

Copy link
Contributor

@joostjager joostjager left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Proto file also needs to be updated for new meaning of this boolean.

Copy link
Contributor

@halseth halseth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is okay to slightly change the RPC behavior. You can argue that the current behavior is a bug.

@bhandras
Copy link
Collaborator Author

Proto file also needs to be updated for new meaning of this boolean.

Thanks for pointing this out. Updated.

@bhandras bhandras requested a review from joostjager December 18, 2019 14:42
@bhandras bhandras added this to the 0.9.0 milestone Dec 18, 2019
@bhandras bhandras added bug fix code health Related to code commenting, refactoring, and other non-behaviour improvements labels Dec 18, 2019
@bhandras bhandras self-assigned this Dec 18, 2019
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Doesn't this reduce the effect of this test, because we rely on the code under test to provides us with the expectations?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd argue that it actually makes the test more strict as before we relied on FetchAllInvoices too. Ideally expected results should be constant and possbily included in the test itself rather then generated/selected etc.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The removal of FetchAllInvoices is good, but it is now replaced by something that uses the code under test to generate the expectations. This test is testing whether the filtering is correct. I'd say we don't need to do the deep equal on the exact invoice contents for that. It is tested elsewhere already. For me, building a list of payment hashes here and asserting on it in the various filter cases is enough.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not convinced that this is not a test relaxation (compared to what it was before). Maybe it's more correct this way though. PTAL.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn't the updatedInvoice still used as an expectation now?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, it is changed in a later commit. Generally we don't want to touch the same code in one pr twice. Switching them around as you had before lowers the summed delta of all commits.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, that was a rebase mistake on my end. Corrected.

@bhandras bhandras force-pushed the i1404 branch 2 times, most recently from 8ee9c05 to fa44f40 Compare December 18, 2019 15:32
@bhandras bhandras requested a review from joostjager December 18, 2019 15:33
@bhandras bhandras force-pushed the i1404 branch 5 times, most recently from 4424e32 to 557108f Compare December 19, 2019 08:45
Copy link
Contributor

@joostjager joostjager left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commit order seems to be not optimal in this revision?

@bhandras
Copy link
Collaborator Author

Commit order seems to be not optimal in this revision?

Yeah, forgot to tamper with dates when rebasing interactively. It was in good order just GitHub orders by date. Fixed now.

@bhandras bhandras requested a review from joostjager December 19, 2019 14:04
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still using updatedInvoice for expectation?

@bhandras bhandras force-pushed the i1404 branch 4 times, most recently from 9df7188 to d523d9e Compare December 19, 2019 16:15
@bhandras
Copy link
Collaborator Author

bhandras commented Dec 19, 2019

After much discussion about what to include in the PR and how to change tests @joostjager and I decided to go back in time somewhat and just revert to the version where we keep FetchAllInvoices intact as removal and fixes of the tests where this call is used introduces a lot of unrelated changes and should be done in subsequent cleanup PRs.

@bhandras bhandras requested a review from joostjager December 19, 2019 16:30
This commit removes random invoice state selection when testing
FetchAllInvoicesWithPaymentHash.
This commits builds on top of PR lightningnetwork#3694 to further clarify invoice
state by defining pending invoices as the ones which are not
settled or canceled. Automatic cancellation of expired invoices
makes this possbile. While this change only directly affects
ChannelDB, users of the listinvoices RPC will receive actual
pending invoices when pending_only flag is set.
@joostjager
Copy link
Contributor

@bhandras it is ok to merge without a fresh approval from @halseth, because in the end the pr was reverted to the revision that he originally acked.

@joostjager joostjager merged commit 5ea6315 into lightningnetwork:master Dec 19, 2019
@bhandras bhandras deleted the i1404 branch January 21, 2020 19:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug fix code health Related to code commenting, refactoring, and other non-behaviour improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants