Skip to content

Core: Delete orphaned merged DV files on cache invalidation and commit failure - #17309

Open
wangyum wants to merge 1 commit into
apache:mainfrom
wangyum:fix/orphaned-merged-dv-puffin-file
Open

Core: Delete orphaned merged DV files on cache invalidation and commit failure#17309
wangyum wants to merge 1 commit into
apache:mainfrom
wangyum:fix/orphaned-merged-dv-puffin-file

Conversation

@wangyum

@wangyum wangyum commented Jul 21, 2026

Copy link
Copy Markdown
Member

What

MergingSnapshotProducer.mergeDVs() writes a Puffin file each time duplicate DVs are merged. When the delete manifest cache is invalidated (by a subsequent addDeletes call) or the commit fails, only the manifest files were deleted — the Puffin files were left orphaned on disk.

Why

After #15006, mergeDVs() writes a Puffin file (e.g. merged-dvs-<snapshotId>-<n>.puffin) and caches the resulting delete manifests. Two cleanup paths missed the Puffin files:

  1. Cache invalidation (newDeleteFilesAsManifests): deleted the cached manifests but not the Puffin files they referenced.
  2. Commit failure (cleanUncommittedAppends): deleted uncommitted delete manifests but not their Puffin files.

The orphaned files can only be removed by remove_orphan_files. No data corruption — read correctness is preserved.

How

Track merged DV file locations in a cachedMergedDVLocations field (paralleling cachedNewDeleteManifests) and delete them in both cleanup paths. The location is resolved through the FileIO so its scheme matches DeleteFile.location() (e.g. stripping a file: URI scheme).

Tests

  • testMergedDVPuffinFileCleanedUpOnCacheInvalidation — verifies the Puffin file is deleted when a subsequent addDeletes invalidates the cache, and the committed snapshot has a correctly merged DV.
  • testMergedDVPuffinFileCleanedUpOnCommitFailure — verifies the Puffin file is deleted when the commit fails (via injected failCommits).

Closes #17307


AI Disclosure

  • Model: GLM 5.2
  • Platform/Tool: opencode
  • Human Oversight: fully reviewed
  • Prompt Summary: Fix orphaned merged DV Puffin file on cache invalidation and commit failure, with tests covering both paths.

@github-actions github-actions Bot added the core label Jul 21, 2026
@wangyum

wangyum commented Jul 21, 2026

Copy link
Copy Markdown
Member Author

cc @amogh-jahagirdar @rdblue

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Core: Orphaned merged DV file on cache invalidation and commit failure

1 participant