Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

routing: persistent mission control #3164

Merged
merged 9 commits into from Aug 2, 2019

Conversation

@joostjager
Copy link
Collaborator

commented Jun 5, 2019

This PR adds persistence to mission control by storing the raw payment results and reprocessing them on startup.

@joostjager joostjager changed the title routing: persistent mission control routing: persistent mission control [wip] Jun 5, 2019

@joostjager joostjager force-pushed the joostjager:persistent-mc branch 13 times, most recently from 06997c0 to 222e395 Jun 12, 2019

@joostjager joostjager force-pushed the joostjager:persistent-mc branch 4 times, most recently from f5a1dc7 to 215f2d3 Jun 18, 2019

@joostjager joostjager force-pushed the joostjager:persistent-mc branch 6 times, most recently from 8681348 to 69cc375 Jun 20, 2019

@joostjager joostjager requested a review from halseth Jul 19, 2019

@joostjager joostjager force-pushed the joostjager:persistent-mc branch from e10b03b to 6207be7 Jul 19, 2019

@wpaulino wpaulino removed their request for review Jul 20, 2019

@wpaulino wpaulino added this to the 0.8.0 milestone Jul 25, 2019

joostjager added some commits Jun 26, 2019

routing: extract timestamp from applying payment result
This commit prepares for timestamps being supplied from the database for
processing of historical payment results.
routing: define payment result
This commit groups together all payment result data. It is a preparation
for historical payment results being retrieved from the database.

@joostjager joostjager force-pushed the joostjager:persistent-mc branch 5 times, most recently from 84d123a to c9d2d8e Jul 29, 2019

joostjager added some commits Jun 14, 2019

routing/missioncontrol_store.go Outdated Show resolved Hide resolved
for b.numRecords >= b.maxRecords {
cursor := bucket.Cursor()
cursor.First()
if err := cursor.Delete(); err != nil {

This comment has been minimized.

Copy link
@cfromknecht

cfromknecht Jul 30, 2019

Collaborator

in the past i think we observed a performance issue in calling First() then Delete() repetitively, since it actually results in a quadratic operation. fortunately i think we only expect this to be called once here, but something to keep in mind in the future. @Roasbeef was this not one of the root causes in that migration performance issue?

This comment has been minimized.

Copy link
@joostjager

joostjager Jul 31, 2019

Author Collaborator

Ok interesting. Yes, it is expected to be called only once.

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Aug 2, 2019

Member

IIRC, in that case, we would have repeated calls to Delete(), then First(), as Delete would move the cursor one record after the deleted key. We ended up resolving it by using a two pass approach: 974ec02#diff-99f3013528876f55a0e46ff7014e8a19

@joostjager joostjager force-pushed the joostjager:persistent-mc branch from 5a4b1b3 to 7e7b620 Jul 31, 2019

@joostjager joostjager requested a review from cfromknecht Jul 31, 2019

@cfromknecht
Copy link
Collaborator

left a comment

LGTM 💯

@Roasbeef
Copy link
Member

left a comment

LGTM 🥨

for b.numRecords >= b.maxRecords {
cursor := bucket.Cursor()
cursor.First()
if err := cursor.Delete(); err != nil {

This comment has been minimized.

Copy link
@Roasbeef

Roasbeef Aug 2, 2019

Member

IIRC, in that case, we would have repeated calls to Delete(), then First(), as Delete would move the cursor one record after the deleted key. We ended up resolving it by using a two pass approach: 974ec02#diff-99f3013528876f55a0e46ff7014e8a19

@Roasbeef Roasbeef merged commit 7767eec into lightningnetwork:master Aug 2, 2019

2 checks passed

continuous-integration/travis-ci/pr The Travis CI build passed
Details
coverage/coveralls Coverage increased (+0.04%) to 60.752%
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.