-
Notifications
You must be signed in to change notification settings - Fork 624
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Unload obsolete memtries post block processing #11657
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #11657 +/- ##
==========================================
- Coverage 71.65% 71.61% -0.05%
==========================================
Files 787 787
Lines 161063 161104 +41
Branches 161063 161104 +41
==========================================
- Hits 115413 115373 -40
- Misses 40597 40684 +87
+ Partials 5053 5047 -6
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
That sounds right. However, still looks hacky that we are doing this check for every single block, while the shard "status" changes only once per epoch. I have a feeling that we can call Also nit - if the function is called |
Currently we remove obsolete memtries only when we start state sync for a new shard: https://github.com/near/nearcore/blob/master/chain/client/src/client.rs#L2507-L2519 However, it does not unload obsolete memtries in all cases. For example, if we tracked shards [0, 1, 2], and then we track shards [0, 1] only, shard 2 won't be unloaded, because we did not start a new state sync. Although it is not critical, trie for shard 2 could occupy memory forever. This PR calls the routine to unload obsolete memtries as a block post-processing step too.
Currently we remove obsolete memtries only when we start state sync for a new shard: https://github.com/near/nearcore/blob/master/chain/client/src/client.rs#L2507-L2519
However, it does not unload obsolete memtries in all cases.
For example, if we tracked shards [0, 1, 2], and then we track shards [0, 1] only, shard 2 won't be unloaded, because we did not start a new state sync.
Although it is not critical, trie for shard 2 could occupy memory forever.
This PR calls the routine to unload obsolete memtries as a block post-processing step too.