[SharovBot] fix: handle snapshot infohash mismatch on restart by Giulio2002 · Pull Request #19316 · erigontech/erigon

Giulio2002 · 2026-02-19T14:45:13Z

Problem

Fixes #19128

On restart after a fresh sync, the downloader encounters a fatal error:

downloading snapshot v1.1-018000-018500-bodies.seg (infohash X): snapshot already loaded with different infohash: Y

This happens because background processes (e.g. recsplit) can rebuild snapshot files while the node is running, changing the file's infohash. On restart:

The torrent is loaded from disk with the new infohash (from the updated .torrent file)
The preverified list still references the old infohash
getExistingSnapshotTorrent returns a fatal error on mismatch

Fix

In addPreverifiedSnapshotForDownload, when we detect an infohash mismatch between an already-loaded torrent and the preverified list, we now skip the preverified download and keep the existing local torrent instead of returning an error. The local file is valid - it was loaded from disk and passed completeness checks.

This aligns with the existing test TestChangeInfoHashOfSameFile which documents this exact scenario and expects the existing torrent to be kept.

Testing

All existing downloader tests pass
TestChangeInfoHashOfSameFile specifically covers this scenario

When a snapshot file is rebuilt in the background (e.g. by recsplit) between restarts, its infohash changes. On the next restart, the torrent is loaded from disk with the new infohash, but the preverified list still references the old one. Previously this caused a fatal error: 'snapshot already loaded with different infohash'. Now we gracefully skip the preverified download and keep the existing local torrent, since the local file is valid. Fixes #19128

Giulio2002 · 2026-02-19T17:25:07Z

[SharovBot] 🔄 Restarted failing CI workflow: 22186493427 (All tests - tests-mac-linux ubuntu-24.04)

anacrolix

There's possibly a few issues here I think:

Snapshots should only be requested for download during an initial sync. A node should not be in a state where an initial sync is required and already have snapshots present that didn't match (there's currently a minor case where consecutive initial syncs that don't complete can disagree which a snapshots reset fixes).
If snapshot file contents change, they should be removed from the downloader first. This is a major bug if it's still occurring. New snapshots should have different names, and if that's not possible, the snapshot should be removed from the download by name before adding the replacement.

I think the former is the larger concern here, why hasn't the node already completed the initial sync before something like this could happen.

I created #19435 for this problem.

anacrolix

Hm that's not a blocking review, I just want to check the points are sound before merging. As above I think this indicates a broken use case, but this is a good workaround.

…ent and issue reference When addPreverifiedSnapshotForDownload encounters an infohash mismatch between a loaded torrent and the preverified list, keep the existing local torrent (no crash). Improve the log message to be explicit that the preverified download is being skipped, and add a comment pointing to the root-cause tracking issue #19435 (AddTorrentsFromDisk ordering relative to initial sync). The proper structural fix (running initial sync before AddTorrentsFromDisk so preverified TOML hashes always win) is tracked in issue #19435.

Giulio2002 · 2026-02-24T01:37:24Z

[SharovBot] 🔧 Addressed @anacrolix feedback — updated comment and log message to be explicit about the tradeoff, and added a reference to #19435 as the proper structural fix.

On the 'remove first' suggestion: I attempted a 'remove existing, add preverified' approach, but TestChangeInfoHashOfSameFile explicitly tests the opposite — it documents that a node with a local snapshot should keep its version when a new preverified hash is released (e.g. re-compressed release). That test would fail with the remove-first approach.

On the root cause concern: Agreed that the real fix is running initial sync before AddTorrentsFromDisk so preverified TOML hashes always win. That structural change is what #19435 tracks. This PR is a minimal stopgap to prevent the fatal crash while that deeper fix is implemented — with an explicit code comment now pointing to #19435.

Giulio2002 · 2026-02-24T03:33:18Z

[SharovBot] 🔄 Restarted failing CI workflow 22332838279 (tests-windows)

Giulio2002 · 2026-02-24T09:13:08Z

Oh @anacrolix AI is pretty bad at PR desc. basically a failure there should not lead to a total crash. a WARN is enough imo

anacrolix · 2026-02-24T11:29:29Z

Damn I shouldn't have hit update branch now it's running everything again

Giulio2002 · 2026-02-24T11:33:10Z

[SharovBot] ✅ Auto-merge enabled — approved by erigontech org member @anacrolix

Giulio2002 · 2026-02-24T12:33:16Z

[SharovBot] 🔄 CI rerun already in progress for failing tests-mac-linux (macos-15) (run 22348856962) — monitoring for auto-merge.

Giulio2002 · 2026-02-24T13:33:13Z

[SharovBot] 🔄 Restarted failing CI workflow 22348856962 — TestShutterBlockBuilding on macos-15 is unrelated to this PR's changes (snapshot infohash fix); appears to be a flaky integration test.

Giulio2002 · 2026-02-24T14:33:43Z

[SharovBot] 🔄 Restarted cancelled Unit tests workflow 22348856935 (linux (macos-latest) was cancelled, blocking auto-merge).

## Problem Fixes #19128 On restart after a fresh sync, the downloader encounters a fatal error: ``` downloading snapshot v1.1-018000-018500-bodies.seg (infohash X): snapshot already loaded with different infohash: Y ``` This happens because background processes (e.g. recsplit) can rebuild snapshot files while the node is running, changing the file's infohash. On restart: 1. The torrent is loaded from disk with the **new** infohash (from the updated .torrent file) 2. The preverified list still references the **old** infohash 3. `getExistingSnapshotTorrent` returns a fatal error on mismatch ## Fix In `addPreverifiedSnapshotForDownload`, when we detect an infohash mismatch between an already-loaded torrent and the preverified list, we now **skip the preverified download** and keep the existing local torrent instead of returning an error. The local file is valid - it was loaded from disk and passed completeness checks. This aligns with the existing test `TestChangeInfoHashOfSameFile` which documents this exact scenario and expects the existing torrent to be kept. ## Testing - All existing downloader tests pass - `TestChangeInfoHashOfSameFile` specifically covers this scenario

@Giulio2002

## Summary - Moves `AddTorrentsFromDisk` call from `initDownloader` (at startup, before any sync) to an `afterSnapshotDownload` callback invoked once the snapshot sync stage completes on the first sync cycle. - This eliminates false-positive incomplete torrent reports that occurred because we were registering on-disk snapshots before the sync stage had a chance to complete them. - The `afterDownload` callback is threaded through `StageSnapshotsCfg`, `NewDefaultStages`, and `NewPipelineStages` with a `nil` default so callers that don't need it are unaffected. - Incomplete snapshot warnings after this callback are now flagged as unexpected (sync just finished), with a comment noting the preverified-set edge case that remains unresolved. ## Background Issue #19435 first identified this problem: `AddTorrentsFromDisk` was being called before initial sync status was checked, causing old or stale `.torrent` files on disk to interfere with the sync stage's ability to fetch the correct (preverified) versions of snapshots. PR #19316 by @Giulio2002 addressed a related symptom — converting the fatal "snapshot already loaded with different infohash" error in `addPreverifiedSnapshotForDownload` into a warning — but this was treating a symptom rather than the root cause. Other related changes to this area: - PR #18056 — "Final downloader cleanup": Changed `AddTorrentsFromDisk` to only add completed snapshots (anything incomplete is assumed a mistake), and removed it as a default at startup. - PR #18270 — "Quick fix for --downloader.verify assertion": Fixed a panic in `addCompleteTorrentFromMetainfo` when called from `AddTorrentsFromDisk` on already-completed files (#18165). - PR #15043 — "Fix a collection of downloader, snapshot sync and torrent related issues": Earlier broad fix addressing how disk-loaded torrents interact with preverified snapshots. - PR #16655 — "Require torrents added from disk to complete on `downloader.verify`": Enforced completion check for disk-loaded torrents during verify. @mh0lt — please take a look, particularly at the scenario where extra snapshots land on disk that weren't part of the sync set (e.g. after an upgrade or downgrade that changes the preverified set). --------- Co-authored-by: Alex Sharov <AskAlexSharov@gmail.com>

@Giulio2002

## Summary - Moves `AddTorrentsFromDisk` call from `initDownloader` (at startup, before any sync) to an `afterSnapshotDownload` callback invoked once the snapshot sync stage completes on the first sync cycle. - This eliminates false-positive incomplete torrent reports that occurred because we were registering on-disk snapshots before the sync stage had a chance to complete them. - The `afterDownload` callback is threaded through `StageSnapshotsCfg`, `NewDefaultStages`, and `NewPipelineStages` with a `nil` default so callers that don't need it are unaffected. - Incomplete snapshot warnings after this callback are now flagged as unexpected (sync just finished), with a comment noting the preverified-set edge case that remains unresolved. ## Background Issue #19435 first identified this problem: `AddTorrentsFromDisk` was being called before initial sync status was checked, causing old or stale `.torrent` files on disk to interfere with the sync stage's ability to fetch the correct (preverified) versions of snapshots. PR #19316 by @Giulio2002 addressed a related symptom — converting the fatal "snapshot already loaded with different infohash" error in `addPreverifiedSnapshotForDownload` into a warning — but this was treating a symptom rather than the root cause. Other related changes to this area: - PR #18056 — "Final downloader cleanup": Changed `AddTorrentsFromDisk` to only add completed snapshots (anything incomplete is assumed a mistake), and removed it as a default at startup. - PR #18270 — "Quick fix for --downloader.verify assertion": Fixed a panic in `addCompleteTorrentFromMetainfo` when called from `AddTorrentsFromDisk` on already-completed files (#18165). - PR #15043 — "Fix a collection of downloader, snapshot sync and torrent related issues": Earlier broad fix addressing how disk-loaded torrents interact with preverified snapshots. - PR #16655 — "Require torrents added from disk to complete on `downloader.verify`": Enforced completion check for disk-loaded torrents during verify. @mh0lt — please take a look, particularly at the scenario where extra snapshots land on disk that weren't part of the sync set (e.g. after an upgrade or downgrade that changes the preverified set). --------- Co-authored-by: Alex Sharov <AskAlexSharov@gmail.com>

Giulio2002 requested review from AskAlexSharov and sudeepdino008 as code owners February 19, 2026 14:45

Giulio2002 requested a review from taratorio February 19, 2026 14:45

taratorio requested a review from anacrolix February 20, 2026 03:46

Giulio2002 enabled auto-merge (squash) February 23, 2026 23:25

anacrolix mentioned this pull request Feb 24, 2026

downloader: Adding existing snapshots #19435

Open

anacrolix requested changes Feb 24, 2026

View reviewed changes

anacrolix reviewed Feb 24, 2026

View reviewed changes

Merge branch 'main' into fix/snapshot-infohash-mismatch-on-restart

f408123

anacrolix approved these changes Feb 24, 2026

View reviewed changes

Giulio2002 merged commit a3b76e9 into main Feb 24, 2026
31 of 35 checks passed

Giulio2002 deleted the fix/snapshot-infohash-mismatch-on-restart branch February 24, 2026 15:05

anacrolix mentioned this pull request Mar 10, 2026

downloader: add torrents from disk after snapshot sync stage #19712

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SharovBot] fix: handle snapshot infohash mismatch on restart#19316

[SharovBot] fix: handle snapshot infohash mismatch on restart#19316
Giulio2002 merged 3 commits intomainfrom
fix/snapshot-infohash-mismatch-on-restart

Giulio2002 commented Feb 19, 2026

Uh oh!

Giulio2002 commented Feb 19, 2026

Uh oh!

anacrolix left a comment

Uh oh!

anacrolix left a comment

Uh oh!

Giulio2002 commented Feb 24, 2026

Uh oh!

Giulio2002 commented Feb 24, 2026

Uh oh!

Giulio2002 commented Feb 24, 2026 •

edited

Loading

Uh oh!

anacrolix commented Feb 24, 2026

Uh oh!

Giulio2002 commented Feb 24, 2026

Uh oh!

Giulio2002 commented Feb 24, 2026

Uh oh!

Giulio2002 commented Feb 24, 2026

Uh oh!

Giulio2002 commented Feb 24, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Giulio2002 commented Feb 19, 2026

Problem

Fix

Testing

Uh oh!

Giulio2002 commented Feb 19, 2026

Uh oh!

anacrolix left a comment

Choose a reason for hiding this comment

Uh oh!

anacrolix left a comment

Choose a reason for hiding this comment

Uh oh!

Giulio2002 commented Feb 24, 2026

Uh oh!

Giulio2002 commented Feb 24, 2026

Uh oh!

Giulio2002 commented Feb 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

anacrolix commented Feb 24, 2026

Uh oh!

Giulio2002 commented Feb 24, 2026

Uh oh!

Giulio2002 commented Feb 24, 2026

Uh oh!

Giulio2002 commented Feb 24, 2026

Uh oh!

Giulio2002 commented Feb 24, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Giulio2002 commented Feb 24, 2026 •

edited

Loading