Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merge SHiP stability fixes & log splitting & memory usage improvements #628

Merged
merged 60 commits into from
Mar 2, 2023

Conversation

heifner
Copy link
Member

@heifner heifner commented Jan 9, 2023

Merges #638 on top of #577 & #599. Also includes some additional misc. cleanup.

Addresses the double memory used when starting from a snapshot via merge of #577.

This PR backports EOSIO/eos#9277 and EOSIO/eos#9821 to support supports the trace and chain state log spliting in state history plugin.

Some new options for state history plugin are added:

  • state-history-stride: split the state history log files when the block number is the multiple of the stride. When the stride is reached, the current history log and index will be renamed '*-history--.log/index' and a new current history log and index will be created with the most recent blocks. All files following this format will be used to construct an extended history log.
  • max-retained-history-files: the maximum number of state history log file groups to retain so that the blocks in those files can be queried. When the number is reached, the oldest log files would be moved to archive dir or deleted if the archive dir is empty. The retained log files should not be manipulated by users.
  • state-history-retained-dir: the location of the state history retained directory (absolute path or relative to state-history dir).
  • state-history-archive-dir: the location of the state history archive directory (absolute path or relative to state-history dir). If the value is empty string, log files beyond the retained limit will be deleted. All files in the archive directory are completely under user's control, i.e. they won't be accessed by nodeos anymore.

Verified this branch also fixes #592, #203

huangminghuang added a commit that referenced this pull request Feb 10, 2023
…ship-log

# Conflicts:
#	libraries/chain/include/eosio/chain/log_catalog.hpp
#	libraries/chain/include/eosio/chain/log_data_base.hpp
#	libraries/chain/include/eosio/chain/log_index.hpp
@huangminghuang huangminghuang changed the base branch from main to huangminghuang/block_log February 10, 2023 22:40
Base automatically changed from huangminghuang/block_log to main February 28, 2023 23:57
@heifner heifner added the documentation Improvements or additions to documentation label Mar 1, 2023
Copy link
Contributor

@ScottBailey ScottBailey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please address comments in:

  • libraries/chain/include/eosio/chain/log_catalog.hpp
  • plugins/state_history_plugin/include/eosio/state_history_plugin/session.hpp

Thanks!

Copy link
Contributor

@ScottBailey ScottBailey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation OCI Work exclusive to OCI team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3.2 state_history_plugin crashes under heavy load
5 participants