db/state: RangeAsOf DB iterator starts from files.EndTxNum()#20360
Merged
sudeepdino008 merged 5 commits intomainfrom Apr 8, 2026
Merged
db/state: RangeAsOf DB iterator starts from files.EndTxNum()#20360sudeepdino008 merged 5 commits intomainfrom
sudeepdino008 merged 5 commits intomainfrom
Conversation
AskAlexSharov
approved these changes
Apr 6, 2026
sudeepdino008
force-pushed
the
sudeep/rao_db_files
branch
from
6e4e30f to
5175ab6
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Adjusts HistoryRoTx.RangeAsOf to avoid redundant DB iteration over the tx range already covered by frozen files, enabling safe pruning of DB entries in that range.
Changes:
- Start the DB-side
RangeAsOfiterator frommax(startTxNum, files.EndTxNum())to skip the frozen-file-covered range. - Add a regression test that exercises
RangeAsOfbehavior after pruning DB entries within the file range.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
db/state/history.go |
Shifts DB iterator start txNum to max(startTxNum, ht.iit.files.EndTxNum()) to avoid reading prunable/redundant DB data covered by files. |
db/state/history_test.go |
Adds TestRangeAsOf_DBIteratorSkipsFileRange to validate RangeAsOf results remain correct after pruning DB entries within the file range. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
sudeepdino008
force-pushed
the
sudeep/rao_db_files
branch
from
5175ab6 to
649e664
Compare
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
AskAlexSharov
added a commit
that referenced
this pull request
Apr 9, 2026
The optimization to start the DB iterator at max(startTxNum, files.EndTxNum()) assumed files always cover all history entries below EndTxNum. In practice, files may be incomplete (during segment merges, partial rebuilds, etc.), causing the DB iterator to skip real history entries. When history misses a key, DomainRoTx.RangeAsOf falls back to the latest state, incorrectly returning current values for historical queries (e.g., accounts appearing at block 1M that were created later). This caused flaky failures in debug_accountRange and debug_storageRangeAt RPC integration tests on mainnet.
Closed
This was referenced Apr 10, 2026
github-merge-queue Bot
pushed a commit
that referenced
this pull request
Apr 11, 2026
…st (#20444) Fixes #20355 - Skip DB entries whose step falls within the file range (`step.ToTxNum < files.EndTxNum()`) in both the initial DB cursor push and the advance loop — they are never loaded into the heap - Matches the "don't query DB for data in files" pattern from #20360, #20361, #20362 - Add `TestDomain_IteratePrefix_PrefersFilesOverDB` that simulates partial lexicographic prune and verifies file values win Follow-up: #20474 applies the same fix to `DomainLatestIterFile.initCursorMDBX()` and `advanceInFiles()` ## Test plan - [x] New test passes with fix, fails without it - [x] `make lint` clean - [x] `make erigon integration` builds
sudeepdino008
added a commit
that referenced
this pull request
Apr 13, 2026
- In `RangeAsOf`, the DB iterator (`HistoryRangeAsOfDB`) now starts from `max(startTxNum, files.EndTxNum())` instead of `startTxNum` directly - This avoids reading DB entries within the file range, which are redundant (served by `HistoryRangeAsOfFiles`) and prunable - Added `TestRangeAsOf_DBIteratorSkipsFileRange` to verify correctness after pruning DB entries in the file range Closes #20357 --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Alex Sharov <AskAlexSharov@gmail.com>
sudeepdino008
added a commit
that referenced
this pull request
Apr 13, 2026
…st (#20444) Fixes #20355 - Skip DB entries whose step falls within the file range (`step.ToTxNum < files.EndTxNum()`) in both the initial DB cursor push and the advance loop — they are never loaded into the heap - Matches the "don't query DB for data in files" pattern from #20360, - Add `TestDomain_IteratePrefix_PrefersFilesOverDB` that simulates partial lexicographic prune and verifies file values win Follow-up: #20474 applies the same fix to `DomainLatestIterFile.initCursorMDBX()` and `advanceInFiles()` - [x] New test passes with fix, fails without it - [x] `make lint` clean - [x] `make erigon integration` builds
sudeepdino008
added a commit
that referenced
this pull request
Apr 13, 2026
- In `RangeAsOf`, the DB iterator (`HistoryRangeAsOfDB`) now starts from `max(startTxNum, files.EndTxNum())` instead of `startTxNum` directly - This avoids reading DB entries within the file range, which are redundant (served by `HistoryRangeAsOfFiles`) and prunable - Added `TestRangeAsOf_DBIteratorSkipsFileRange` to verify correctness after pruning DB entries in the file range Closes #20357 --------- Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Alex Sharov <AskAlexSharov@gmail.com>
sudeepdino008
added a commit
that referenced
this pull request
Apr 13, 2026
…st (#20444) Fixes #20355 - Skip DB entries whose step falls within the file range (`step.ToTxNum < files.EndTxNum()`) in both the initial DB cursor push and the advance loop — they are never loaded into the heap - Matches the "don't query DB for data in files" pattern from #20360, - Add `TestDomain_IteratePrefix_PrefersFilesOverDB` that simulates partial lexicographic prune and verifies file values win Follow-up: #20474 applies the same fix to `DomainLatestIterFile.initCursorMDBX()` and `advanceInFiles()` - [x] New test passes with fix, fails without it - [x] `make lint` clean - [x] `make erigon integration` builds
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
RangeAsOf, the DB iterator (HistoryRangeAsOfDB) now starts frommax(startTxNum, files.EndTxNum())instead ofstartTxNumdirectlyHistoryRangeAsOfFiles) and prunableTestRangeAsOf_DBIteratorSkipsFileRangeto verify correctness after pruning DB entries in the file rangeCloses #20357