routing: persistent mission control #3164
Conversation
joostjager
changed the title
joostjager
force-pushed
the
persistent-mc
branch
13 times, most recently
from
06997c0
to
222e395
Compare
Roasbeef
added
database
joostjager
force-pushed
the
persistent-mc
branch
4 times, most recently
from
f5a1dc7
to
215f2d3
Compare
joostjager
force-pushed
the
persistent-mc
branch
6 times, most recently
from
8681348
to
69cc375
Compare
joostjager
force-pushed
the
persistent-mc
branch
from
e10b03b
to
6207be7
Compare
This commit prepares for timestamps being supplied from the database for processing of historical payment results.
This commit groups together all payment result data. It is a preparation for historical payment results being retrieved from the database.
joostjager
force-pushed
the
persistent-mc
branch
5 times, most recently
from
84d123a
to
c9d2d8e
Compare
joostjager
force-pushed
the
persistent-mc
branch
from
c9d2d8e
to
5a4b1b3
Compare
Roasbeef
removed
the
incomplete
| for b.numRecords >= b.maxRecords { | ||
| cursor := bucket.Cursor() | ||
| cursor.First() | ||
| if err := cursor.Delete(); err != nil { |
There was a problem hiding this comment.
in the past i think we observed a performance issue in calling First() then Delete() repetitively, since it actually results in a quadratic operation. fortunately i think we only expect this to be called once here, but something to keep in mind in the future. @Roasbeef was this not one of the root causes in that migration performance issue?
There was a problem hiding this comment.
Ok interesting. Yes, it is expected to be called only once.
There was a problem hiding this comment.
IIRC, in that case, we would have repeated calls to Delete(), then First(), as Delete would move the cursor one record after the deleted key. We ended up resolving it by using a two pass approach: 974ec02#diff-99f3013528876f55a0e46ff7014e8a19
joostjager
force-pushed
the
persistent-mc
branch
from
5a4b1b3
to
7e7b620
Compare
| for b.numRecords >= b.maxRecords { | ||
| cursor := bucket.Cursor() | ||
| cursor.First() | ||
| if err := cursor.Delete(); err != nil { |
There was a problem hiding this comment.
IIRC, in that case, we would have repeated calls to Delete(), then First(), as Delete would move the cursor one record after the deleted key. We ended up resolving it by using a two pass approach: 974ec02#diff-99f3013528876f55a0e46ff7014e8a19
This PR adds persistence to mission control by storing the raw payment results and reprocessing them on startup.