-
Notifications
You must be signed in to change notification settings - Fork 91
config: merge RemoveUntraceableHeaders into RemoveUntraceableBlocks #3922
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Some of statesync module tests are failing due to #3795, will include this issue into this PR, we'll need to implement it anyway soon. |
@roman-khimov, it's still a draft, but you may review the idea of the last commit, we need to discuss this solution. An alternative (presize) solution messes up the code of statesync module. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks like #3795 part of it is a bit premature.
Not yet ready for review, we still need to request trusted block to properly verify next headers. |
A lot of tests are still failing due to changes related to untraceable blocks removal, so still in the process of fixing them. The main logic is implemented and may be reviewed. |
10f918a
to
802c342
Compare
@roman-khimov, let's perform some initial review. One problem marked with TODO is left and some new tests are still needed. All existing tests are finally fixed. |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #3922 +/- ##
==========================================
- Coverage 82.36% 82.31% -0.06%
==========================================
Files 345 347 +2
Lines 48773 48965 +192
==========================================
+ Hits 40173 40305 +132
- Misses 6924 6974 +50
- Partials 1676 1686 +10 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
004e646
to
6cbafb5
Compare
Close #3920. Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
1. Optimize it a bit by searching from the first suitable batch of headers at the DB Search level. 2. Rename it since this method removes the head of header hashes (we'll have code that removes tail of it soon). Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Early stop is not hard to support and it will be useful for untraceable header hashes removal. Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Use `cont` instead of `f` for processing function, otherwise it's not clear what is the return value meaning from the caller's side. Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Light nodes synchronized over NeoFSStateExchange won't have these data anyway, hence it would be nice to have unified behaviour for all light nodes. Old header hashes are not used anyway. Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
f170b8f
to
55a5669
Compare
@roman-khimov just for the record, GH tests are fixed (there was a bug actually, but it revealed only on slow machines), various manual node synchronization scenarios are checked, untraceable blocks removal check is in progress. Ready for review. |
Untraceable blocks removal on my machine with LevelDB and default GCP takes seconds (4-8 seconds) on syncing mainnet node. I'd say it's acceptable, especially comparing with MPT GC:
|
@@ -177,6 +177,12 @@ func (s *Module) Init(currChainHeight uint32) error { | |||
return nil | |||
} | |||
if s.bc.BlockHeight() > p-2*s.syncInterval { | |||
trustedH := s.bc.GetConfig().TrustedHeader.Index | |||
if trustedH > s.bc.BlockHeight() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's a pretty narrow case (s.bc.BlockHeight() > p-2*s.syncInterval
), the node is almost in sync, so to me it should just work. I also doubt we can we end up here without changing TrustedHeight
in the config (from the one originally used). Anyway, if the height is good and trusted (and local DB is trusted by definition) we better just keep going.
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Should be a part of #2085, discovered during digging into the problem of inlined structures, ref. go-yaml/yaml#125. Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
If batch of headers consists of stale headers only, then don't try to add any of them. Originally introduced by c3e73c5. Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Do not request the whole batch of headers if syncing via P2PStateExchangeExtensions or via NeoFSStateSyncExtensions. Instead, request headers starting from some trusted height. Introducing trusted header allows to enable tests related to untraceable blocks/headers removal. Close #3795. Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Use LE to log header hash. Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
No functional changes. Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
No functional changes. Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
If corresponding header was stored in the DB we need to verify incoming block hash against this header. Otherwise, we may end up in a corrupted DB and malicious block saved to the database. Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
When submitting block, we need to distinguish if it already exists on the chain or if it's a block from future. Signed-off-by: Anna Shaleva <shaleva.ann@nspcc.ru>
Close #3920, close #3795.
TODO:
Blocks
mode).HeaderHashes
initialization on node restart after statesync interruption.