Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor: abstract store and remove all db.Atomic usage #5021

Merged
merged 3 commits into from
Jan 30, 2023

Conversation

icorderi
Copy link
Contributor

Summary

  • this adds a top level store and three root functions for operating with the db
  • direct usage of db.Pair and/or db.Accessor for the trackerdb/catchpoints has been removed.
  • Transaction (atomic read+write), Snapshot (isolated reads), Batch (all or nothing writes)

Upcoming PRs:

  • We will attempt to remove all usages of Transaction in favor of either Snapshot or Batch.
  • We will change the *sql.Tx exposed in this functions to be only the already defined "reader/writer" abstarctions.

Test Plan

Existing tests.

@codecov
Copy link

codecov bot commented Jan 30, 2023

Codecov Report

Merging #5021 (9468203) into master (1ad3db9) will decrease coverage by 0.05%.
The diff coverage is 52.38%.

@@            Coverage Diff             @@
##           master    #5021      +/-   ##
==========================================
- Coverage   53.57%   53.53%   -0.05%     
==========================================
  Files         430      431       +1     
  Lines       54091    54120      +29     
==========================================
- Hits        28980    28971       -9     
- Misses      22864    22900      +36     
- Partials     2247     2249       +2     
Impacted Files Coverage Δ
ledger/store/store.go 0.00% <0.00%> (ø)
ledger/store/testing.go 56.09% <0.00%> (-6.07%) ⬇️
ledger/acctupdates.go 69.11% <66.66%> (-0.13%) ⬇️
ledger/catchpointtracker.go 57.92% <66.66%> (-0.57%) ⬇️
ledger/catchupaccessor.go 64.45% <94.73%> (-0.32%) ⬇️
ledger/acctonline.go 78.64% <100.00%> (ø)
ledger/ledger.go 70.09% <100.00%> (-0.10%) ⬇️
ledger/tracker.go 74.26% <100.00%> (-0.85%) ⬇️
ledger/trackerdb.go 67.64% <100.00%> (ø)
ledger/txtail.go 78.52% <100.00%> (-0.15%) ⬇️
... and 7 more

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@@ -1292,7 +1292,7 @@ func TestAcctOnlineVotersLongerHistory(t *testing.T) {
// DB has all the required history tho
var dbOnlineRoundParams []ledgercore.OnlineRoundParamsData
var endRound basics.Round
err = oa.dbs.Rdb.Atomic(func(ctx context.Context, tx *sql.Tx) (err error) {
err = oa.dbs.Batch(func(ctx context.Context, tx *sql.Tx) (err error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Snapshot? AccountsOnlineRoundParams is part of accountsV2Reader

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we will sort out snapshot, batch, transaction on the PR I'm work on on right now based on what methods they end up calling.

The PR I'm working on gets rid of that tx *sql.Tx in the callbacks in favor of using interfaces with the methods we have defined already.

@@ -167,7 +166,7 @@ func (ml *mockLedgerForTracker) fork(t testing.TB) *mockLedgerForTracker {
dbs.Rdb.SetLogger(dblogger)
dbs.Wdb.SetLogger(dblogger)

newLedgerTracker.dbs = dbs
newLedgerTracker.dbs = store.CreateTrackerSQLStore(dbs)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think go prefers Make/New prefix instead of Create for such functions

@algorandskiy algorandskiy merged commit 8970ddf into algorand:master Jan 30, 2023
@@ -1090,7 +1087,7 @@ func (ct *catchpointTracker) generateCatchpointData(ctx context.Context, account
var catchpointWriter *catchpointWriter
start := time.Now()
ledgerGeneratecatchpointCount.Inc(nil)
err = ct.dbs.Rdb.AtomicContext(ctx, func(dbCtx context.Context, tx *sql.Tx) (err error) {
err = ct.dbs.BatchContext(ctx, func(dbCtx context.Context, tx *sql.Tx) (err error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in #5583

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants