client/db: active and archived match buckets #1263
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
chappjc
force-pushed
the
active-matchs
branch
from
1ebfedb
to
b67901d
Compare
chappjc
commented
Nov 3, 2021
JoeGruffins
approved these changes
Nov 3, 2021
chappjc
commented
Nov 3, 2021
| @@ -4673,7 +4673,7 @@ func (c *Core) dbTrackers(dc *dexConnection) (map[order.OrderID]*trackedTrade, e | |||
| for _, dbMatch := range dbMatches { | |||
| // Only trade matches are added to the matches map. Detect and skip | |||
| // cancel order matches, which have an empty Address field. | |||
| if dbMatch.Address == "" { | |||
| if dbMatch.Address == "" { // only correct for maker's cancel match | |||
There was a problem hiding this comment.
This is a sorta redundant check anyway since MatchesForOrder a few lines up should screen them out, but if it fails to do so, then this check is effective for screening out this trade order's cancel match, which was a problem in the past.
chappjc
commented
Nov 3, 2021
Comment on lines
-95
to
-96
| byteEpoch = uint16Bytes(uint16(order.OrderStatusEpoch)) | ||
| byteBooked = uint16Bytes(uint16(order.OrderStatusBooked)) |
There was a problem hiding this comment.
Were used before the orders bucket was split.
chappjc
force-pushed
the
active-matchs
branch
2 times, most recently
from
2fb45ae
to
0ea934e
Compare
buck54321
approved these changes
Nov 8, 2021
Comment on lines
+18
to
+24
| // TODO: consider changing upgradefunc to accept a bbolt.DB so it may create its | ||
| // own transactions. The individual upgrades have to happen in separate | ||
| // transactions because they get too big for a single transaction. Plus we | ||
| // should consider switching certain upgrades from ForEach to a more cumbersome | ||
| // Cursor()-based iteration that will facilitate partitioning updates into | ||
| // smaller batches of buckets. |
Comment on lines
+130
to
+135
| matchesBucket := []byte("matches") | ||
| return reloadMatchProofs(dbtx, skipCancels, matchesBucket) |
There was a problem hiding this comment.
archivedMatchesBucket? Or would that be confusing?
There was a problem hiding this comment.
Just trying to make these older upgrades more robust to future changes to package level variables by defining the bucket/key names as they are known to be at that particular version.
Comment on lines
+302
to
+307
| oldMatchesBucket := []byte("matches") | ||
| newActiveMatchesBucket := []byte("activeMatches") |
There was a problem hiding this comment.
Not using activeMatchesBucket and archivedMatchesBucket because this is better semantics?
There was a problem hiding this comment.
Missed this comment, sorry. Same consideration as other comment - a v7+ upgrade might change the package-level variables, but this v6 upgrade will always need to work with a specific key as it was in v5
chappjc
force-pushed
the
active-matchs
branch
2 times, most recently
from
d5fc628
to
57c65da
Compare
client/db: add v3upgrade to set match retired field for existing matches merge (*trackedTrade).matchIsActive and db.isRetiredMatch There is now one single function with the heuristics for determining a match's activeness, used by both trackedTrade and db.
chappjc
force-pushed
the
active-matchs
branch
from
57c65da
to
7909ec7
Compare
This redefines the v5 client db upgrade to introduce a new active matches bucket instead of creating a Retired flag on each match. The db and upgrade code is modeled after the active/archived orders buckets changes. client/db.MatchIsActive is updated to detect cancel matches as inactive. This is detected as MatchComplete and len(proof.Auth.InitSig) == 0 since cancel matches begin as complete and have no init/redeem data set in the Auth field. The Address field is only empty for the maker/trade match with a cancel, but for the taker/cancel's match with the trade it has to this point contained the canceled maker order's address. This also updates client/core to clear the Address field for both sides of a cancel match. Removing this useless data from a MetaMatch will save a bit of storage space, but previous cancel matches are not updated by the v6 upgrade. Shut down the db tests properly. upgrade logger, log matches upgrade outcome
chappjc
force-pushed
the
active-matchs
branch
from
e2b65e7
to
7a35720
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This builds on #957. @itswisdomagain's commits will be merged without squashing into this. I've added his v5 test db generator app as well, just with an added cancel order match.
This redefines the v6 client db upgrade to introduce a new active matches bucket instead of creating a
Retiredflag on each match.The db and upgrade code is modeled after the active/archived orders buckets changes in #1071, whereby existing matches remain in what is to be the archived bucket, and only active ones are moved into the new active matches bucket.
client/db.MatchIsActiveis updated to detect cancel matches as inactive. This is detected asMatchCompleteandlen(proof.Auth.InitSig) == 0since cancel matches begin as complete and have no init/redeem data set in theAuthfield.The
Addressfield is only empty for the maker/trade match with a cancel, but for the taker/cancel's match with the trade it has to this point contained the canceled maker order's address.This also updates client/core to clear the
Addressfield for both sides of a cancel match because it is pointless to store. Removing this useless data from aMetaMatchwill save a bit of storage space, but previous cancel matches are not updated by the v6 upgrade. Similarly, the server's swapper is updated to not send the maker address with a cancelmsgjson.Match.Shut down the db tests properly.
Maybe 0.3