cp: integrity blk-range, up gql/grpc, stepSize tooling, CursorHeap fix, xsync fix, warmuper fix, logs debug, unused code by AskAlexSharov · Pull Request #20337 · erigontech/erigon

AskAlexSharov · 2026-04-05T09:24:16Z

Cherry-picks from release/3.4 to main:

integrity: blk-range chk to pre-build "changed keys" index before calc state root (integrity: blk-range chk to pre-build "changed keys" index before calc state root #20302)
up gql and grpc (up gql and grpc #20304)
stepSize: all tooling to use tx.Debug().StepSize() instead of DefaultStepSize constant (stepSize: all tooling to use tx.Debug().StepSize() instead of DefaultStepSize constant #20280)
db/state: fix CursorHeap tie-break to prefer RAM over DB over FILE ([r3.4] db/state: fix CursorHeap tie-break to prefer RAM over DB over FILE #20318)
xsync deprecated use fix (xsync deprecated use fix #20331)
warmuper.WaitAndClose: cancel work before wait (warmuper.WaitAndClose: cancel work before wait #20332)
logs: move some Info logs to Debug level (to simplify logs for Users) (logs: move some Info logs to Debug level (to simplify logs for Users) #20329)
db: unused code remove (db: unused code remove #20334)

…c state root (#20302) 10_000x faster `check-commitment-hist-at-blk-range` in-mem index of "changed keys per each block" built once per 10K blocks Problem: each `HistoryKeyTxNumRange()` call it's full-scan of `.ef` file - and now it happening per-block. Solution: call it once per 10K blocks and process results into "blockNum -> changedKeys" idx

…ultStepSize` constant (#20280)

…FILE (#20318) - When `DomainDelPrefix` iterates storage keys during account deletion, it merges RAM, DB, and FILE cursors via a min-heap - RAM and DB cursors both used `endTxNum: math.MaxUint64`, so when the same key existed in both, heap order was undefined - If DB won the tie, the old value was used as `prevVal` in the history entry instead of the current uncommitted RAM value - This caused corrupted history values that propagated to commitment root mismatches Fix: add tie-breaker in `CursorHeap.Less` — when `endTxNum` is equal, prefer `RAM_CURSOR > DB_CURSOR > FILE_CURSOR` **Safety**: `CursorHeap` is used in 3 contexts: - `merge.go` / `deduplicate.go`: only `FILE_CURSOR` — tie-break is a no-op (same type, returns false) - `DomainLatestIterFile` (DB + FILE): DB `endTxNum = step * stepSize` always differs from FILE `endTxNum = item.endTxNum - 1`, so tie-break rarely fires. If it does, DB correctly wins over FILE - `debugIteratePrefixLatest` (RAM + DB + FILE): the bug fix — RAM now correctly wins over DB when both use `MaxUint64` Fixes #20246

before: - seems we never close Warmuper - because WaitAndClose switching atomic. - seems we wait-first and then cancel/close work channel - it's wrong because `WaitAndClose()` called when warmup not needed anymore (commitment calc finished): ``` warmuper.Start() defer warmuper.WaitAndClose() ``` PR: canceling first and then wait gorouines to finish

…#20329)

Copilot

Pull request overview

Cherry-pick bundle from release/3.4 to main, primarily improving integrity/state-history verification performance and aligning tooling with configurable step-size, plus several correctness fixes (CursorHeap ordering, warmuper shutdown) and dependency/log-level updates.

Changes:

Add an in-memory “changed keys per block” index to accelerate check-commitment-hist-at-blk-range, and refactor commitment-history checks to reuse prebuilt indices.
Update tooling to use dynamic step size (tx.Debug().StepSize() / erigondb.toml settings) instead of DefaultStepSize.
Fix CursorHeap tie-break ordering (RAM > DB > FILE), adjust warmuper shutdown ordering, reduce log verbosity, and remove unused code.

Reviewed changes

Copilot reviewed 31 out of 32 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
go.mod	Bumps gqlgen/grpc/x/* and other deps; adds indirect go-yaml; updates erigon-snapshot revision
go.sum	Syncs sums to dependency updates
execution/types/receipt_test.go	Removes unused helpers/imports from receipt tests
execution/stagedsync/exec3_serial.go	Reuses state cache per loop; adjusts when stats are printed/reset
execution/execmodule/exec_module.go	Always passes state cache; actual enablement gated inside SharedDomains
execution/commitment/warmuper.go	Replaces WaitAndClose with CloseAndWait (cancel/close before waiting)
execution/commitment/hex_patricia_hashed.go	Switches deferred shutdown to `CloseAndWait()`
execution/commitment/commitmentdb/commitment_context.go	Alters commitment processing debug logging behavior (currently commented out)
execution/cache/state_cache.go	Makes `PrintStatsAndReset()` safe on nil receiver
db/state/inverted_index.go	Removes unused ef path mask helper
db/state/history.go	Removes unused path mask helpers and deprecated file getter
db/state/execctx/domain_shared.go	`SetStateCache` now respects `dbg.UseStateCache`; removes unused const
db/state/domain.go	Removes unused file mask helpers and unused buffer field
db/state/domain_stream.go	Fixes CursorHeap tie-break to prefer RAM over DB over FILE when endTxNum ties
db/state/domain_stream_test.go	Adds tests covering the new CursorHeap tie-break rules
db/state/aggregator.go	Downgrades some snapshot merge logs from Info to Debug
db/snapcfg/util.go	Adjusts embedded snapshot hash/webseed registry entries (removes some chains)
db/rawdb/rawdbhelpers/rawdbhelpers.go	Adds helper to estimate step-range present in an arbitrary DupSort txNum-keyed table
db/integrity/deref_cache.go	Removes unused cache-entry encoder helper
db/integrity/commitment_integrity.go	Refactors commitment-history checks to build/reuse changed-keys indices; changes blk-range checking strategy and concurrency model
db/integrity/changed_keys_per_block.go	New: builds per-domain changed-keys-per-block in-memory index + TxNum→Block mapper
db/integrity/changed_keys_per_block_test.go	New: unit tests for changed-keys-per-block indexing and TxNum→Block mapping
db/config3/config3.go	Clarifies comments: prefer settings from erigondb.toml / tx.Debug().StepSize()
common/maphash/maphash.go	Updates xsync usage from deprecated MapOf API to Map API
cmd/utils/app/snapshots_cmd.go	Updates integrity command call sites to new commitment-history API signature
cmd/rpcdaemon/graphql/graph/schema.resolvers.go	Regenerated header comment for gqlgen version bump
cmd/rpcdaemon/graphql/graph/generated.go	Large gqlgen-generated update (generic config/state, concurrency helpers, nil-input handling)
cmd/integration/commands/state_history.go	Uses resolved erigondb settings for step size/frozen-file steps when reading history
cmd/integration/commands/reset_state.go	Extends `print_stages` output; uses `tx.Debug().StepSize()` and new DB progress helpers
cmd/integration/commands/commitment.go	Uses `tx.Debug().StepSize()` for commitment reads/benchmarks
cl/transition/impl/eth2/operations.go	Removes unused deposit signature validation helper
.github/workflows/manifest.yml	Updates webseed URLs from erigon31-v1 to erigon34-v1

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- changed_keys_per_block: cast windowLen to int for make/range loop - commitment_integrity: create sampler per-goroutine (Sampler.rng not goroutine-safe); limit concurrency to GOMAXPROCS/4 for memory-heavy windows - rawdbhelpers: guard len>=8 before BigEndian.Uint64 in IdxStepsInDB - warmuper: route Wait() through Close() to use atomic guard, preventing double-close panic - commitment_context: remove dead _ = logPrefix and commented-out log block Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

Copilot reviewed 30 out of 31 changed files in this pull request and generated 6 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- commitment_integrity: remove unused db param from checkCommitmentHistAtBlkWithIdx - aggregator: remove commented-out log lines Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

AskAlexSharov and others added 5 commits April 5, 2026 16:18

up gql and grpc (#20304)

c34c0ac

stepSize: all tooling to use tx.Debug().StepSize() instead of `Defa…

18e2eeb

…ultStepSize` constant (#20280)

xsync deprecated use fix (#20331)

29fedf5

AskAlexSharov requested review from Giulio2002, antonis19, awskii, sudeepdino008 and taratorio as code owners April 5, 2026 09:24

go mod tidy after updating golang.org/x/* deps

680c6f0

AskAlexSharov enabled auto-merge April 5, 2026 09:38

AskAlexSharov added 3 commits April 5, 2026 16:56

rawdbhelpers: add IdxStepsInDB; fix reset_state imports

ac76509

Merge branch 'main' into alex/cp_fixes_35

35ef8fc

save

a16b6e1

AskAlexSharov requested a review from mriccobene as a code owner April 5, 2026 13:37

AskAlexSharov requested a review from JkLondon April 5, 2026 13:46

AskAlexSharov added 3 commits April 5, 2026 21:35

logs: move some Info logs to Debug level (to simplify logs for Users) (…

543709e

…#20329)

db: unused code remove (#20334)

cfd48e2

AskAlexSharov changed the title ~~cp: integrity blk-range, up gql/grpc, stepSize tooling, CursorHeap fix, xsync fix~~ cp: integrity blk-range, up gql/grpc, stepSize tooling, CursorHeap fix, xsync fix, warmuper fix, logs debug, unused code Apr 5, 2026

AskAlexSharov requested review from domiwei, mh0lt and yperbasis as code owners April 5, 2026 14:35

AskAlexSharov requested a review from Copilot April 6, 2026 01:00

Copilot started reviewing on behalf of AskAlexSharov April 6, 2026 01:00 View session

Copilot AI reviewed Apr 6, 2026

View reviewed changes

AskAlexSharov and others added 2 commits April 6, 2026 08:18

Merge remote-tracking branch 'origin/main' into alex/cp_fixes_35

5e17249

AskAlexSharov force-pushed the alex/cp_fixes_35 branch from 18b6d0c to 5e17249 Compare April 6, 2026 01:18

simplify: clarify magic 8 and workerLimit/4 constants

ed1a2bc

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

AskAlexSharov requested a review from Copilot April 6, 2026 02:23

Copilot started reviewing on behalf of AskAlexSharov April 6, 2026 02:24 View session

Copilot AI reviewed Apr 6, 2026

View reviewed changes

Comment thread execution/commitment/warmuper.go

Comment thread db/snapcfg/util.go

Comment thread db/snapcfg/util.go

Comment thread db/state/aggregator.go Outdated

Comment thread db/integrity/commitment_integrity.go Outdated

Comment thread db/integrity/commitment_integrity.go

AskAlexSharov and others added 2 commits April 6, 2026 10:06

address more CR comments on #20337

eee8768

- commitment_integrity: remove unused db param from checkCommitmentHistAtBlkWithIdx - aggregator: remove commented-out log lines Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

save

43d2a9a

AskAlexSharov force-pushed the alex/cp_fixes_35 branch from 04768cf to 43d2a9a Compare April 6, 2026 03:08

AskAlexSharov added 2 commits April 6, 2026 10:11

save

a2e0048

save

98c303d

sudeepdino008 approved these changes Apr 6, 2026

View reviewed changes

AskAlexSharov added this pull request to the merge queue Apr 6, 2026

Merged via the queue into main with commit d6d3518 Apr 6, 2026
36 checks passed

AskAlexSharov deleted the alex/cp_fixes_35 branch April 6, 2026 07:13

AskAlexSharov mentioned this pull request Apr 23, 2026

cp: cherry-pick batch 5 from release/3.4 to main #20759

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

cp: integrity blk-range, up gql/grpc, stepSize tooling, CursorHeap fix, xsync fix, warmuper fix, logs debug, unused code#20337

cp: integrity blk-range, up gql/grpc, stepSize tooling, CursorHeap fix, xsync fix, warmuper fix, logs debug, unused code#20337
AskAlexSharov merged 19 commits intomainfrom
alex/cp_fixes_35

AskAlexSharov commented Apr 5, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

AskAlexSharov commented Apr 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

AskAlexSharov commented Apr 5, 2026 •

edited

Loading