Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

routing: persistent mission control #3164

Merged
merged 9 commits into from Aug 2, 2019

Conversation

@joostjager
Copy link
Collaborator

@joostjager joostjager commented Jun 5, 2019

This PR adds persistence to mission control by storing the raw payment results and reprocessing them on startup.

@joostjager joostjager changed the title routing: persistent mission control routing: persistent mission control [wip] Jun 5, 2019
@joostjager joostjager force-pushed the persistent-mc branch 13 times, most recently from 06997c0 to 222e395 Jun 14, 2019
@joostjager joostjager force-pushed the persistent-mc branch 4 times, most recently from f5a1dc7 to 215f2d3 Jun 19, 2019
@joostjager joostjager force-pushed the persistent-mc branch 6 times, most recently from 8681348 to 69cc375 Jun 27, 2019
@joostjager joostjager requested review from cfromknecht and halseth Jul 19, 2019
@wpaulino wpaulino removed their request for review Jul 20, 2019
@wpaulino wpaulino added this to the 0.8.0 milestone Jul 25, 2019
joostjager added 6 commits Jul 29, 2019
This commit prepares for timestamps being supplied from the database for
processing of historical payment results.
This commit groups together all payment result data. It is a preparation
for historical payment results being retrieved from the database.
@joostjager joostjager force-pushed the persistent-mc branch 5 times, most recently from 84d123a to c9d2d8e Jul 29, 2019
routing/missioncontrol_store.go Outdated Show resolved Hide resolved
Loading
for b.numRecords >= b.maxRecords {
cursor := bucket.Cursor()
cursor.First()
if err := cursor.Delete(); err != nil {
Copy link
Collaborator

@cfromknecht cfromknecht Jul 30, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

in the past i think we observed a performance issue in calling First() then Delete() repetitively, since it actually results in a quadratic operation. fortunately i think we only expect this to be called once here, but something to keep in mind in the future. @Roasbeef was this not one of the root causes in that migration performance issue?

Loading

Copy link
Collaborator Author

@joostjager joostjager Jul 31, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok interesting. Yes, it is expected to be called only once.

Loading

Copy link
Member

@Roasbeef Roasbeef Aug 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC, in that case, we would have repeated calls to Delete(), then First(), as Delete would move the cursor one record after the deleted key. We ended up resolving it by using a two pass approach: 974ec02#diff-99f3013528876f55a0e46ff7014e8a19

Loading

Copy link
Collaborator

@cfromknecht cfromknecht left a comment

LGTM 💯

Loading

Copy link
Member

@Roasbeef Roasbeef left a comment

LGTM 🥨

Loading

for b.numRecords >= b.maxRecords {
cursor := bucket.Cursor()
cursor.First()
if err := cursor.Delete(); err != nil {
Copy link
Member

@Roasbeef Roasbeef Aug 2, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIRC, in that case, we would have repeated calls to Delete(), then First(), as Delete would move the cursor one record after the deleted key. We ended up resolving it by using a two pass approach: 974ec02#diff-99f3013528876f55a0e46ff7014e8a19

Loading

@Roasbeef Roasbeef merged commit 7767eec into lightningnetwork:master Aug 2, 2019
2 checks passed
Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

5 participants