Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

blockchain: Make indexing asynchronous. #1470

Closed
davecgh opened this issue Sep 26, 2018 · 3 comments · Fixed by #2219
Closed

blockchain: Make indexing asynchronous. #1470

davecgh opened this issue Sep 26, 2018 · 3 comments · Fixed by #2219
Assignees

Comments

@davecgh
Copy link
Member

davecgh commented Sep 26, 2018

Currently, all of the index updates happen during the block connection path and in the same database transaction as all other chain-related updates. This was originally done to ensure atomicity. However, since the indexes are merely indexing immutable blockchain data, failure to write the most recent data is completely recoverable via the reconcile path at startup, and thus, all indexing should ideally be done via a separate goroutine.

Also, as it is currently written, by indexing in the same thread as the block validation and connection code, it slows down block validation as compared to not having indexing enabled. Block validation and connection needs to be as fast as possible, so this is not desirable.

This will require taking the time to really understand the concurrency model used in the blockchain package to ensure that unclean shutdown recover properly and do no leave the database in an inconsistent state.

@davecgh davecgh changed the title [blockchain] Make indexing more cache friendly. [blockchain] Make indexing asynchronous. Sep 26, 2018
@dnldd
Copy link
Member

dnldd commented Oct 1, 2018

I'd like to take this on.

@davecgh
Copy link
Member Author

davecgh commented Oct 1, 2018

I should've commented, but this one is already taken.

@kLkA
Copy link
Contributor

kLkA commented Dec 15, 2018

@davecgh I've published #1553 PR for utxoview replacement with spend journal as we discussed it would be better to have as independent feature first

@davecgh davecgh changed the title [blockchain] Make indexing asynchronous. blockchain: Make indexing asynchronous. Dec 28, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants