Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Oom fixes 2022.01.19 #3709

Merged
merged 17 commits into from
Jan 27, 2022
Merged

Oom fixes 2022.01.19 #3709

merged 17 commits into from
Jan 27, 2022

Conversation

iulianpascalau
Copy link
Contributor

@iulianpascalau iulianpascalau commented Jan 19, 2022

  • made the db pointer inside the storers implementation nil whenever we call Close or Destroy. Added nil checks on all usages of the db pointer.
  • peer accounts trie snapshot for shard nodes so the pruning storer for the peer accounts will not cause OOMs
  • minor storers refactor: used context instead of chan

…we call Close or Destroy. Added nil checks on all usages of the db pointer.
@iulianpascalau iulianpascalau changed the base branch from master to development January 19, 2022 10:17
@codecov
Copy link

codecov bot commented Jan 19, 2022

Codecov Report

Merging #3709 (5c7f6c7) into development (9e8e91a) will increase coverage by 0.00%.
The diff coverage is 85.62%.

Impacted file tree graph

@@             Coverage Diff              @@
##           development    #3709   +/-   ##
============================================
  Coverage        73.78%   73.79%           
============================================
  Files              589      589           
  Lines            76218    76312   +94     
============================================
+ Hits             56241    56316   +75     
- Misses           15530    15545   +15     
- Partials          4447     4451    +4     
Impacted Files Coverage Δ
process/block/shardblock.go 67.07% <0.00%> (-0.66%) ⬇️
storage/leveldb/serialActions.go 73.52% <66.66%> (-11.48%) ⬇️
storage/leveldb/leveldbSerial.go 81.86% <92.30%> (+1.97%) ⬆️
storage/pruning/pruningStorer.go 72.90% <93.10%> (+1.02%) ⬆️
storage/leveldb/leveldb.go 81.56% <94.73%> (+0.10%) ⬆️
factory/blockProcessorCreator.go 79.52% <100.00%> (+0.02%) ⬆️
state/peerAccountsDB.go 87.64% <100.00%> (+2.11%) ⬆️
storage/leveldb/common.go 83.33% <100.00%> (+4.76%) ⬆️
storage/pruning/fullHistoryPruningStorer.go 77.77% <100.00%> (+2.33%) ⬆️
storage/pruning/triePruningStorer.go 66.66% <100.00%> (ø)
... and 5 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fac376d...5c7f6c7. Read the comment docs.

@sasurobert sasurobert self-requested a review January 21, 2022 10:23
sasurobert
sasurobert previously approved these changes Jan 21, 2022
go.mod Show resolved Hide resolved
storage/leveldb/common.go Show resolved Hide resolved
bogdan-rosianu
bogdan-rosianu previously approved these changes Jan 21, 2022
storage/leveldb/leveldb.go Show resolved Hide resolved
storage/leveldb/leveldbSerial.go Show resolved Hide resolved
storage/leveldb/leveldb.go Outdated Show resolved Hide resolved
sasurobert
sasurobert previously approved these changes Jan 26, 2022
- added several tests, added minor pruning storer fixes
Copy link
Collaborator

@gabi-vuls gabi-vuls left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

System test passed.

@iulianpascalau iulianpascalau merged commit 67f77b6 into development Jan 27, 2022
@iulianpascalau iulianpascalau deleted the oom-fixes-2022.01.19 branch January 27, 2022 10:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants