Skip to content

snapshotfetch: write appdb meta.json on fetch --import success#127

Merged
zrbecker merged 1 commit into
mainfrom
fetch-import-appdb-meta
May 12, 2026
Merged

snapshotfetch: write appdb meta.json on fetch --import success#127
zrbecker merged 1 commit into
mainfrom
fetch-import-appdb-meta

Conversation

@zrbecker
Copy link
Copy Markdown
Collaborator

Summary

Standalone malcom snapshot import writes <appdb>/meta.json after the import succeeds (in cli/snapshotimport/run.go:268-278), carrying chain_id, height, imported_at, source_snapshot_hash_hex, and db_backend. Downstream commands — malcom verify, malcom bootstrap, malcom bootstrap heal — read this file to default --chain and --height; without it operators have to pass both flags explicitly.

The pipelined fetch --import flow never wrote that file. cli/snapshotfetch/pipeline.go drove snapshotimport.ImportParallel directly and then jumped straight to verify, skipping the appdb-meta write.

Fix

  • pipelineState captures snapDir in onDownloadReady.
  • New pipelineState.writeAppDBMeta(now time.Time) error reads the snapshot's meta.json (for hash_hex), builds an AppDBMeta, and calls snapshotimport.WriteAppDBMeta.
  • cli/snapshotfetch/run.go invokes it between pipeline.finalize and runPostImportVerify on the success path. Failure to write the meta returns ExitDiskFailed.

Test plan

  • TestPipelineWriteAppDBMeta — seeds a fake snapshot meta.json with a known hash_hex, runs the writer, and round-trips through snapshotimport.ReadAppDBMeta to confirm ChainID, Height, SourceSnapshotHashHex, DBBackend, and ImportedAt are all written correctly.
  • Two error-path tests: missing snapDir, missing snapshot meta.json. Both surface a clear error rather than blowing up later in the pipeline.
  • go test ./... -race -count=1 — full module green.
  • go vet ./... && go build ./... — clean.

Impact

After this lands, fetch --import outputs are interchangeable with the two-step fetch + import outputs — same files, same downstream defaults. Operators running malcom verify --appdb <out> or malcom bootstrap --appdb <out> no longer have to pass --chain and --height redundantly.

Standalone `malcom snapshot import` writes <appdb>/meta.json after
the import succeeds (cli/snapshotimport/run.go:268-278), carrying
chain_id, height, imported_at, source_snapshot_hash_hex, and
db_backend. Downstream commands (`verify`, `bootstrap`) read this
file to default --chain and --height; without it operators have
to pass both flags explicitly.

The pipelined `fetch --import` flow never wrote that file. The
cli/snapshotfetch pipeline drove `snapshotimport.ImportParallel`
directly and then jumped to verify, skipping the appdb-meta write.

Fix: capture the snapshot dir in pipelineState.onDownloadReady,
and add pipelineState.writeAppDBMeta that reads the snapshot's
meta.json (for the hash_hex), constructs an AppDBMeta, and calls
snapshotimport.WriteAppDBMeta. run.go invokes it between
pipeline.finalize and runPostImportVerify on the success path.

Tests:

- TestPipelineWriteAppDBMeta seeds a snapshot meta.json with a
  known hash and verifies the appdb meta.json round-trips
  through snapshotimport.ReadAppDBMeta with the expected fields.
- Missing-snapDir and missing-snapshot-meta variants exercise
  the error paths so the failure surfaces with context rather
  than blowing up later.
@zrbecker zrbecker merged commit 63249d9 into main May 12, 2026
1 check passed
@zrbecker zrbecker deleted the fetch-import-appdb-meta branch May 12, 2026 20:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant