Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: introduce init_tip_hash setting for indexer to skip previous block synchronization #4247

Merged

Conversation

EthanYuan
Copy link
Collaborator

@EthanYuan EthanYuan commented Nov 27, 2023

What problem does this PR solve?

Problem Summary:

CKB-indexer supports building indexes only for interested blocks. This can be done by setting block_filter before this PR. But the filter can only traverse the block from 0 and for the uninterested block also need to save its Header(BlockNumber, &'a Byte32, bool).

What is changed and how it works?

This pr introduces the init_tip_hash setting. The init tip is effective as long as it is set higher than the current height of the indexing head (reboot required), as the starting height of the indexing. This provides the following benefits:

  1. synchronization can be started instantly from any height block (by block hash)
  2. the previous block does not need to save any data

What's Changed:

  • Added commented out configuration item to ckb.toml template (as an example)
  • Implement apply_init_tip function, which is called in IndexerService::spawn_poll
  • The reset-data --indexer option is supported.

Related changes

This feature will also be used for index-r:

Check List

Tests

  • Manual test
    • general: the indexer can skip the synchronization of the block before the init tip, and the init tip block only saves the block Header(BlockNumber, &'a Byte32, bool)
    • the init tip is effective as long as it is set higher than the current height of the indexing head (reboot required)
    • when the header corresponding to the init_tip_hash is not found, it will attempt to continue searching, and the indexing service of the indexer will not start

Release note

Title Only: Include only the PR title in the release note.

@EthanYuan EthanYuan requested a review from a team as a code owner November 27, 2023 07:47
@EthanYuan EthanYuan requested review from zhangsoledad and removed request for a team November 27, 2023 07:47
@eval-exec eval-exec added the t:enhancement Type: Feature, refactoring. label Nov 27, 2023
@EthanYuan EthanYuan requested a review from quake November 27, 2023 09:48
@EthanYuan EthanYuan self-assigned this Nov 28, 2023
@EthanYuan EthanYuan force-pushed the indexer-set-init-tip-hash branch 2 times, most recently from 3734d9f to 0b588cd Compare November 30, 2023 14:06
@EthanYuan EthanYuan changed the title feat: introduce init tip setting for indexer to skip previous block synchronization [WIP]feat: introduce init tip setting for indexer to skip previous block synchronization Dec 12, 2023
@EthanYuan EthanYuan changed the title [WIP]feat: introduce init tip setting for indexer to skip previous block synchronization feat: introduce init tip setting for indexer to skip previous block synchronization Dec 15, 2023
@EthanYuan EthanYuan changed the title feat: introduce init tip setting for indexer to skip previous block synchronization feat: introduce init_tip_hash setting for indexer to skip previous block synchronization Dec 15, 2023
@EthanYuan EthanYuan force-pushed the indexer-set-init-tip-hash branch 4 times, most recently from 6531253 to ebda8e9 Compare December 16, 2023 00:50
resource/ckb.toml Outdated Show resolved Hide resolved
@zhangsoledad zhangsoledad added this pull request to the merge queue Jan 11, 2024
Merged via the queue into nervosnetwork:develop with commit 88a70a6 Jan 11, 2024
31 checks passed
@doitian doitian mentioned this pull request Jan 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
t:enhancement Type: Feature, refactoring.
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

None yet

4 participants