Skip to content

feat(flatkv): fix zero storage pruning during flatkv migration#3577

Merged
blindchaser merged 2 commits into
mainfrom
yiren/prune-zero-storage-migration-safe
Jun 11, 2026
Merged

feat(flatkv): fix zero storage pruning during flatkv migration#3577
blindchaser merged 2 commits into
mainfrom
yiren/prune-zero-storage-migration-safe

Conversation

@blindchaser

Copy link
Copy Markdown
Contributor

Summary

Make EVM zero-storage-slot pruning safe to run while a flatkv migration is in progress. The prune loop in PruneZeroStorageSlots previously decided deletions from the iterator's surfaced value. During a flatkv migration the EVM store is a composite/migration store backed by two engines (memiavl + flatkv): point reads (Get/Has) are served by the authoritative routed read path, but iteration is a merged view stitched across both backends by the owning composite store. These two paths can diverge mid-migration, so trusting iterator.Value() could prune a slot that is logically non-zero (live) — corrupting state and producing a storage-digest / apphash mismatch. This change routes the prune decision through the same logical read the EVM itself trusts, so the iterator is used only as a candidate-key source.

  • x/evm/keeper/storage_cleanup.go: PruneZeroStorageSlots now decides deletion from store.Get(key) (the routed logical read) instead of iterator.Value(). The iterator output is treated purely as a list of candidate keys; the all-zero check is evaluated against the authoritative value. Adds a val != nil guard so keys that are logically absent (e.g. a tombstone surfaced by the merged iterator) are skipped rather than re-deleted. This guarantees a slot is pruned only when its authoritative value is all-zero, eliminating the merged-view data-loss path while keeping pruning enabled throughout migrate_evm.

Test plan

  • x/evm/keeper/storage_cleanup_test.go:
    • TestPruneZeroStorageSlots_DecidesDeletionFromRoutedGet (new): drives the real PruneZeroStorageSlots against an injected store whose iterator deliberately lies about Value() while Get returns the truth, reproducing the migration merged-view divergence. Asserts a live slot (routed Get non-zero, iterator shows zero) survives, and a dead slot (routed Get zero, iterator shows non-zero) is pruned. Verified to fail on the pre-fix code (live slot wrongly pruned) and pass on the fix, so it locks in the production behavior change. Uses kvStoreOverrideMultiStore (overrides a single store key), iteratorValueLyingStore (truthful point reads, lying iterator values), and lyingIterator (per-key Value() override).
    • TestPruneZeroStorageSlots (existing): unchanged; continues to cover the normal (non-migration) path, batch limits, checkpoint advance, and checkpoint reset on iterator exhaustion.
  • sei-db/state_db/sc/composite/store_migration_test.go:
    • TestComposite_MigrateEVM_PruneZeroStorageSlotsDuringMigration (new): characterizes the composite store the fix relies on. Seeds zero/non-zero slots, reopens in MigrateEVM, prunes via routed Get, and asserts pruned keys are logically absent (via Get) and absent from the composite iterator, while non-zero slots survive. Continues through migration completion and a final EVMMigrated reopen, asserting version/root-hash continuity (preFlipVersion, preFlipHash) and flatkv lt-hash integrity (VerifyLtHash) at each stage.

Use routed logical reads before pruning zero storage candidates so migrate_evm can keep pruning enabled without trusting a merged iterator view alone.
@cursor

cursor Bot commented Jun 11, 2026

Copy link
Copy Markdown

PR Summary

Medium Risk
Touches EVM state deletion logic during migration; wrong behavior could corrupt storage or apphash, but the change narrows deletes to the authoritative read path and is heavily tested.

Overview
Makes EVM zero-storage pruning safe during flatkv MigrateEVM by basing delete decisions on routed reads, not merged iterator values.

PruneZeroStorageSlots now uses store.Get(key) (with a val != nil guard) to decide whether a slot is all-zero and should be deleted; the iterator is only used to enumerate candidate keys. That matches the migration case where iteration can disagree with authoritative Get/Has across memiavl and flatkv.

New coverage locks this in: a keeper test injects a store whose iterator lies about Value() while Get stays correct, and a composite integration test prunes via routed Get mid-migration, through migration completion, and after EVMMigrated reopen, checking logical absence, iterator visibility, and flatkv hash continuity.

Reviewed by Cursor Bugbot for commit 9e7751d. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions

github-actions Bot commented Jun 11, 2026

Copy link
Copy Markdown

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedJun 11, 2026, 5:06 PM

@codecov

codecov Bot commented Jun 11, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 58.35%. Comparing base (9e65d8e) to head (9e7751d).

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3577      +/-   ##
==========================================
- Coverage   59.18%   58.35%   -0.84%     
==========================================
  Files        2214     2144      -70     
  Lines      183402   175178    -8224     
==========================================
- Hits       108549   102226    -6323     
+ Misses      65047    63812    -1235     
+ Partials     9806     9140     -666     
Flag Coverage Δ
sei-chain-pr 72.10% <100.00%> (?)
sei-db 70.41% <ø> (ø)
sei-db-state-db ?
sei-db-state-db-pr 69.50% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
x/evm/keeper/storage_cleanup.go 89.39% <100.00%> (ø)

... and 116 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@blindchaser blindchaser requested a review from cody-littley June 11, 2026 17:52
@blindchaser blindchaser changed the title feat(flatkv): Make zero storage pruning migration-safe feat(flatkv): make zero storage pruning during flatkv migration Jun 11, 2026
@blindchaser blindchaser added this pull request to the merge queue Jun 11, 2026
Merged via the queue into main with commit 5603c4a Jun 11, 2026
61 checks passed
@blindchaser blindchaser deleted the yiren/prune-zero-storage-migration-safe branch June 11, 2026 19:12
@blindchaser blindchaser changed the title feat(flatkv): make zero storage pruning during flatkv migration feat(flatkv): fix zero storage pruning during flatkv migration Jun 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants