Skip to content

produce optimal merge ranges for state files#19146

Merged
sudeepdino008 merged 3 commits intomainfrom
merge_better
Feb 13, 2026
Merged

produce optimal merge ranges for state files#19146
sudeepdino008 merged 3 commits intomainfrom
merge_better

Conversation

@sudeepdino008
Copy link
Member

@sudeepdino008 sudeepdino008 commented Feb 12, 2026 •
edited
Loading

summary

  • Change >= to > in all three findMergeRange implementations (Domain, II, History) so that a later file with the same from but larger to replaces the current best, producing the largest power-of-2-aligned merge in a single pass
  • Guard iit.mergeFiles call in HistoryRoTx.mergeFiles with r.index.needMerge to avoid panic when only history needs merging (index already merged means from=0, to=0 which hits the startTxNum == endTxNum assert)
  • Update test expectations and comments to match new behavior

context

  • some offline commands like stage_exec or commitment rebuild will produce bunch of smaller files which then need to be merged e,g 0.16, 16-32,32-48, 48-64
  • previous logic will merge first 2 files, then the 3 files.
  • this PR changes that to merge all files directly, reducing amount spent in merging files.

@sudeepdino008 sudeepdino008 changed the title db/state: fix findMergeRange to select largest aligned merge range optimal merge ranges for state file Feb 12, 2026
@sudeepdino008 sudeepdino008 changed the title optimal merge ranges for state file optimal merge ranges for state files Feb 12, 2026
@sudeepdino008 sudeepdino008 changed the title optimal merge ranges for state files produce optimal merge ranges for state files Feb 12, 2026
@sudeepdino008 sudeepdino008 merged commit a43290b into main Feb 13, 2026
27 checks passed
@sudeepdino008 sudeepdino008 deleted the merge_better branch February 13, 2026 10:04
@AskAlexSharov
Copy link
Collaborator

AskAlexSharov commented Feb 15, 2026 •
edited
Loading

i saw:

[INFO] [02-15|11:14:27.589] [snapshots] state merge done accounts(val=640-672, hist=672-688, idx=accounts672-688), storage(val=640-672, hist=672-688, idx=storage672-688), code(val=672-688, hist=672-688, idx=code672-688), commitment(val=640-672, hist=640-672, idx=commitment640-672), receipt(val=640-672, hist=640-672, idx=receipt640-672), rcache(val=640-672, hist=640-672, idx=rcache640-672), logaddrs(640-672), logtopics(640-672), tracesfrom(640-672), tracesto(640-672)
[DBUG] [02-15|11:14:42.795] [agg] merge                              collate_workers=2 merge_workers=1 compress_workers=16
[INFO] [02-15|11:14:42.795] [snapshots] merge state accounts(val=640-704, hist=640-704, idx=accounts640-704), storage(val=640-704, hist=640-704, idx=storage640-704), code(val=640-704, hist=640-704, idx=code640-704), commitment(val=640-704, hist=640-704, idx=commitment640-704), receipt(val=640-704, hist=640-704, idx=receipt640-704), rcache(val=640-704, hist=640-704, idx=rcache640-704), logaddrs(640-704), logtopics(640-704), tracesfrom(640-704), tracesto(640-704)

when executed mainnet from 0

accounts(val=640-672 was done, but then accounts(val=640-704 started.
maybe it's bug?

ls *acc*kv
v2.0-accounts.0-512.kv    v2.0-accounts.647-648.kv  v2.0-accounts.658-659.kv  v2.0-accounts.669-670.kv  v2.0-accounts.680-681.kv  v2.0-accounts.691-692.kv  v2.0-accounts.702-703.kv  v2.0-accounts.713-714.kv  v2.0-accounts.724-725.kv
v2.0-accounts.512-640.kv  v2.0-accounts.648-649.kv  v2.0-accounts.659-660.kv  v2.0-accounts.670-671.kv  v2.0-accounts.681-682.kv  v2.0-accounts.692-693.kv  v2.0-accounts.703-704.kv  v2.0-accounts.714-715.kv  v2.0-accounts.725-726.kv
v2.0-accounts.640-641.kv  v2.0-accounts.649-650.kv  v2.0-accounts.660-661.kv  v2.0-accounts.671-672.kv  v2.0-accounts.682-683.kv  v2.0-accounts.693-694.kv  v2.0-accounts.704-705.kv  v2.0-accounts.715-716.kv  v2.0-accounts.726-727.kv
v2.0-accounts.640-672.kv  v2.0-accounts.650-651.kv  v2.0-accounts.661-662.kv  v2.0-accounts.672-673.kv  v2.0-accounts.683-684.kv  v2.0-accounts.694-695.kv  v2.0-accounts.705-706.kv  v2.0-accounts.716-717.kv  v2.0-accounts.727-728.kv
v2.0-accounts.640-704.kv  v2.0-accounts.651-652.kv  v2.0-accounts.662-663.kv  v2.0-accounts.673-674.kv  v2.0-accounts.684-685.kv  v2.0-accounts.695-696.kv  v2.0-accounts.706-707.kv  v2.0-accounts.717-718.kv  v2.0-accounts.728-729.kv
v2.0-accounts.641-642.kv  v2.0-accounts.652-653.kv  v2.0-accounts.663-664.kv  v2.0-accounts.674-675.kv  v2.0-accounts.685-686.kv  v2.0-accounts.696-697.kv  v2.0-accounts.707-708.kv  v2.0-accounts.718-719.kv  v2.0-accounts.729-730.kv
v2.0-accounts.642-643.kv  v2.0-accounts.653-654.kv  v2.0-accounts.664-665.kv  v2.0-accounts.675-676.kv  v2.0-accounts.686-687.kv  v2.0-accounts.697-698.kv  v2.0-accounts.708-709.kv  v2.0-accounts.719-720.kv
v2.0-accounts.643-644.kv  v2.0-accounts.654-655.kv  v2.0-accounts.665-666.kv  v2.0-accounts.676-677.kv  v2.0-accounts.687-688.kv  v2.0-accounts.698-699.kv  v2.0-accounts.709-710.kv  v2.0-accounts.720-721.kv
v2.0-accounts.644-645.kv  v2.0-accounts.655-656.kv  v2.0-accounts.666-667.kv  v2.0-accounts.677-678.kv  v2.0-accounts.688-689.kv  v2.0-accounts.699-700.kv  v2.0-accounts.710-711.kv  v2.0-accounts.721-722.kv
v2.0-accounts.645-646.kv  v2.0-accounts.656-657.kv  v2.0-accounts.667-668.kv  v2.0-accounts.678-679.kv  v2.0-accounts.689-690.kv  v2.0-accounts.700-701.kv  v2.0-accounts.711-712.kv  v2.0-accounts.722-723.kv
v2.0-accounts.646-647.kv  v2.0-accounts.657-658.kv  v2.0-accounts.668-669.kv  v2.0-accounts.679-680.kv  v2.0-accounts.690-691.kv  v2.0-accounts.701-702.kv  v2.0-accounts.712-713.kv  v2.0-accounts.723-724.kv

@sudeepdino008
Copy link
Member Author

sudeepdino008 commented Feb 15, 2026

i saw:

[INFO] [02-15|11:14:27.589] [snapshots] state merge done accounts(val=640-672, hist=672-688, idx=accounts672-688), storage(val=640-672, hist=672-688, idx=storage672-688), code(val=672-688, hist=672-688, idx=code672-688), commitment(val=640-672, hist=640-672, idx=commitment640-672), receipt(val=640-672, hist=640-672, idx=receipt640-672), rcache(val=640-672, hist=640-672, idx=rcache640-672), logaddrs(640-672), logtopics(640-672), tracesfrom(640-672), tracesto(640-672)
[DBUG] [02-15|11:14:42.795] [agg] merge                              collate_workers=2 merge_workers=1 compress_workers=16
[INFO] [02-15|11:14:42.795] [snapshots] merge state accounts(val=640-704, hist=640-704, idx=accounts640-704), storage(val=640-704, hist=640-704, idx=storage640-704), code(val=640-704, hist=640-704, idx=code640-704), commitment(val=640-704, hist=640-704, idx=commitment640-704), receipt(val=640-704, hist=640-704, idx=receipt640-704), rcache(val=640-704, hist=640-704, idx=rcache640-704), logaddrs(640-704), logtopics(640-704), tracesfrom(640-704), tracesto(640-704)

when executed mainnet from 0

accounts(val=640-672 was done, but then accounts(val=640-704 started. maybe it's bug?

ls *acc*kv
v2.0-accounts.0-512.kv    v2.0-accounts.647-648.kv  v2.0-accounts.658-659.kv  v2.0-accounts.669-670.kv  v2.0-accounts.680-681.kv  v2.0-accounts.691-692.kv  v2.0-accounts.702-703.kv  v2.0-accounts.713-714.kv  v2.0-accounts.724-725.kv
v2.0-accounts.512-640.kv  v2.0-accounts.648-649.kv  v2.0-accounts.659-660.kv  v2.0-accounts.670-671.kv  v2.0-accounts.681-682.kv  v2.0-accounts.692-693.kv  v2.0-accounts.703-704.kv  v2.0-accounts.714-715.kv  v2.0-accounts.725-726.kv
v2.0-accounts.640-641.kv  v2.0-accounts.649-650.kv  v2.0-accounts.660-661.kv  v2.0-accounts.671-672.kv  v2.0-accounts.682-683.kv  v2.0-accounts.693-694.kv  v2.0-accounts.704-705.kv  v2.0-accounts.715-716.kv  v2.0-accounts.726-727.kv
v2.0-accounts.640-672.kv  v2.0-accounts.650-651.kv  v2.0-accounts.661-662.kv  v2.0-accounts.672-673.kv  v2.0-accounts.683-684.kv  v2.0-accounts.694-695.kv  v2.0-accounts.705-706.kv  v2.0-accounts.716-717.kv  v2.0-accounts.727-728.kv
v2.0-accounts.640-704.kv  v2.0-accounts.651-652.kv  v2.0-accounts.662-663.kv  v2.0-accounts.673-674.kv  v2.0-accounts.684-685.kv  v2.0-accounts.695-696.kv  v2.0-accounts.706-707.kv  v2.0-accounts.717-718.kv  v2.0-accounts.728-729.kv
v2.0-accounts.641-642.kv  v2.0-accounts.652-653.kv  v2.0-accounts.663-664.kv  v2.0-accounts.674-675.kv  v2.0-accounts.685-686.kv  v2.0-accounts.696-697.kv  v2.0-accounts.707-708.kv  v2.0-accounts.718-719.kv  v2.0-accounts.729-730.kv
v2.0-accounts.642-643.kv  v2.0-accounts.653-654.kv  v2.0-accounts.664-665.kv  v2.0-accounts.675-676.kv  v2.0-accounts.686-687.kv  v2.0-accounts.697-698.kv  v2.0-accounts.708-709.kv  v2.0-accounts.719-720.kv
v2.0-accounts.643-644.kv  v2.0-accounts.654-655.kv  v2.0-accounts.665-666.kv  v2.0-accounts.676-677.kv  v2.0-accounts.687-688.kv  v2.0-accounts.698-699.kv  v2.0-accounts.709-710.kv  v2.0-accounts.720-721.kv
v2.0-accounts.644-645.kv  v2.0-accounts.655-656.kv  v2.0-accounts.666-667.kv  v2.0-accounts.677-678.kv  v2.0-accounts.688-689.kv  v2.0-accounts.699-700.kv  v2.0-accounts.710-711.kv  v2.0-accounts.721-722.kv
v2.0-accounts.645-646.kv  v2.0-accounts.656-657.kv  v2.0-accounts.667-668.kv  v2.0-accounts.678-679.kv  v2.0-accounts.689-690.kv  v2.0-accounts.700-701.kv  v2.0-accounts.711-712.kv  v2.0-accounts.722-723.kv
v2.0-accounts.646-647.kv  v2.0-accounts.657-658.kv  v2.0-accounts.668-669.kv  v2.0-accounts.679-680.kv  v2.0-accounts.690-691.kv  v2.0-accounts.701-702.kv  v2.0-accounts.712-713.kv  v2.0-accounts.723-724.kv

quite likely existing behavior made more prominent from the change? you have surrounding logs?

@AskAlexSharov AskAlexSharov mentioned this pull request Feb 19, 2026
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@JkLondon JkLondon JkLondon approved these changes

@AskAlexSharov AskAlexSharov Awaiting requested review from AskAlexSharov AskAlexSharov is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@sudeepdino008 @AskAlexSharov @JkLondon