iterateChangedRecent: skip DB entries within file range#20361
iterateChangedRecent: skip DB entries within file range#20361AskAlexSharov merged 3 commits intomainfrom
Conversation
There was a problem hiding this comment.
Pull request overview
Fixes an inconsistency in HistoryRange when the requested range spans both frozen segment files and the DB by ensuring the DB iterator does not start inside the file-covered txNum range (which may be pruned).
Changes:
- Adjust
iterateChangedRecentto start the DB scan atmax(fromTxNum, files.EndTxNum()), aligning behavior withHistoryKeyTxNumRange. - Add a regression test that prunes DB entries within the file range and verifies
HistoryRangestill returns complete results across the file/DB boundary.
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
| db/state/history.go | Skips DB iteration for txNums already covered by frozen files by raising the DB iterator start txNum to files.EndTxNum(). |
| db/state/history_test.go | Adds a regression test that prunes DB entries inside the file-covered range and validates HistoryRange output remains correct. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| // Prune DB entries [0, filesEnd) — these are covered by files. | ||
| rwTx, err := db2.BeginRw(ctx) | ||
| require.NoError(err) |
There was a problem hiding this comment.
The write transaction from db2.BeginRw(ctx) isn’t guarded by defer rwTx.Rollback(). If any require.* fails before the commit, the tx can remain open and leak resources / affect later test cleanup. Add a rollback defer immediately after BeginRw (Commit will make it a no-op).
| require.NoError(err) | |
| require.NoError(err) | |
| defer rwTx.Rollback() |
…st (#20444) Fixes #20355 - Skip DB entries whose step falls within the file range (`step.ToTxNum < files.EndTxNum()`) in both the initial DB cursor push and the advance loop — they are never loaded into the heap - Matches the "don't query DB for data in files" pattern from #20360, #20361, #20362 - Add `TestDomain_IteratePrefix_PrefersFilesOverDB` that simulates partial lexicographic prune and verifies file values win Follow-up: #20474 applies the same fix to `DomainLatestIterFile.initCursorMDBX()` and `advanceInFiles()` ## Test plan - [x] New test passes with fix, fails without it - [x] `make lint` clean - [x] `make erigon integration` builds
Closes #20356 - Adjust `fromTxNum` for the DB iterator in `iterateChangedRecent` to `max(fromTxNum, files.EndTxNum())`, matching the existing pattern in `HistoryKeyTxNumRange` - Prevents reading prunable DB entries that are already covered by segment files - Add `TestHistory_IterateChangedRecent_SkipsFileRange`: builds files, prunes DB entries within the file range, verifies `HistoryRange` spanning both files and DB still returns correct results
Closes #20356 - Adjust `fromTxNum` for the DB iterator in `iterateChangedRecent` to `max(fromTxNum, files.EndTxNum())`, matching the existing pattern in `HistoryKeyTxNumRange` - Prevents reading prunable DB entries that are already covered by segment files - Add `TestHistory_IterateChangedRecent_SkipsFileRange`: builds files, prunes DB entries within the file range, verifies `HistoryRange` spanning both files and DB still returns correct results
Closes #20356
fromTxNumfor the DB iterator initerateChangedRecenttomax(fromTxNum, files.EndTxNum()), matching the existing pattern inHistoryKeyTxNumRangeTestHistory_IterateChangedRecent_SkipsFileRange: builds files, prunes DB entries within the file range, verifiesHistoryRangespanning both files and DB still returns correct results