-
Notifications
You must be signed in to change notification settings - Fork 787
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Experimental ledger pruning #2881
Merged
SergiySW
merged 125 commits into
nanocurrency:develop
from
SergiySW:store/pruning_experimental
Nov 6, 2020
Merged
Experimental ledger pruning #2881
SergiySW
merged 125 commits into
nanocurrency:develop
from
SergiySW:store/pruning_experimental
Nov 6, 2020
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Configuration and considerations - This version of ledger pruning is for experimental use only: - Enable optional ledger pruning with node CLI --debug_prune for manual pruning & node launch flag --enable_pruning for automatic node background actions. - Pruning rules can be configured in [node.experimental] config with “max_pruning_age” (Time in seconds to limit for blocks age after pruning. Also period for automatic node pruning task, default 1 day) & “max_pruning_depth” (Limit for full blocks in chain after pruning, default 0 - unlimited). By default all confirmed blocks older than 1 day can be pruned, except genesis block & last confirmed block in each account chain. - Last confirmed block in account chain & all unconfirmed blocks cannot be pruned from ledger. Pruning is relying on confirmation height, so confirmation height reset or too low online_weight_minimum config can cause issues. - Pruned blocks hashes are located in database table “pruned” & can be used by node internally. There is no specific RPC call to check pruned block existence, but we can add it if necessary. RPC, CLI and other changes - Several RPC calls are modified to support pruned blocks related to main request block. - RPC “history”: if previous block is pruned, “amount” field is not returned for all blocks, “account” field is not returned for state blocks, also state block type/subtype is “unknown” . If source is pruned, then “account” field is not returned for receive/open legacy & for receive/open state blocks. - RPC “block_info”/“blocks_info”: if previous block is pruned, “amount” field is not returned. RPC “block_count”: for nodes with enabled pruning returns 2 new fields: “full” blocks & “pruned” blocks. “full” + “pruned” = total “blocks”. - Block count for CLI/RPC/telemetry is sum of full blocks & pruned blocks. Other notes - Node is aggressively pruning ledger during initial bootstrap (until hardcoded block count) to limit node disk space usage, pruning config settings are ignored. If you want to avoid this behavior, finish full bootstrap before pruning start. - --enable_pruning flag & config option “enable_voting” are mutually exclusive to prevent representatives start with pruned ledger. Pruning is not available for any representatives. Additional updates being reviewed - Include pruned count in telemetry.
wezrule
previously approved these changes
Oct 14, 2020
This was referenced Oct 16, 2020
wezrule
previously approved these changes
Oct 29, 2020
wezrule
approved these changes
Nov 6, 2020
Closed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Labels
cli
Changes related to the Command Line Interface
documentation
This item indicates the need for or supplies updated or expanded documentation
enhancement
major
This item indicates the need for or supplies a major or notable change
rpc
Changes related to Remote Procedure Calls
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is the initial, experimental implementation of pruning, related to #1094. Node ledger pruning is designed to safely discard parts of the ledger. Now ledger size can be reduced from 23.8 GB to 7.5GB after vacuum. And from 18.8 GB to 5.2 GB after database rebuild.
Configuration and considerations
--debug_prune
for manual pruning & node launch flag--enable_pruning
for automatic node background actions.RPC, CLI and other changes
Other notes
--enable_pruning
flag & config option “enable_voting” are mutually exclusive to prevent representatives start with pruned ledger. Pruning is not available for any representatives ([Pruning] Pruning configuration settings #2947).Additional updates being reviewed