Skip to content

fix(blob): gate blob unlink on the delete/update committing (#1364) - #1369

Merged
kriszyp merged 1 commit into
mainfrom
kris/blob-unlink-recheck-1364
Jun 18, 2026
Merged

fix(blob): gate blob unlink on the delete/update committing (#1364)#1369
kriszyp merged 1 commit into
mainfrom
kris/blob-unlink-recheck-1364

Conversation

@kriszyp

@kriszyp kriszyp commented Jun 18, 2026

Copy link
Copy Markdown
Member

Fixes #1364.

Summary

removeEntry deleted a record's blob files up front, before (and independent of) the record removal actually committing. The fix defers the blob deletion until the removal's commit promise resolves, so the unlink only happens once the delete is durable.

Purpose

Second of the two orphan-producing paths in the JJill preprod replication incident. When the Cache expiration scan's transaction is force-committed without a given delete — or any removal aborts/version-conflicts — the record survives while its blob files are unlinked. That orphaned reference makes a later copy/replication stream hit sendBlob() ENOENT and stall permanently. (Receiver-side tolerance for a missing blob is #1353; this stops the orphan being created.)

Approach

store.remove() returns a promise that resolves when the write commits (truthy on commit, false on a conditional-version miss) and rejects on abort — the same promise the eviction scan already awaits. So removeEntry now schedules the blob unlink only after that promise resolves committed, and skips it otherwise. Falls back to immediate deletion when remove() is synchronous (already durable).

This can only ever skip an unlink, never lose data; a skipped unlink leaves a reclaimable orphaned file (swept by cleanupOrphans).

Where to look

  • resources/RecordEncoder.tsremoveEntry. This is the whole change. It works for both the LMDB and RocksDB eviction paths because the removal is queued into the transaction that evict() later commits, so its promise resolves at that commit — no change to evict() itself.

Notes

🤖 Generated by Claude (Opus 4.8, 1M context).

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@claude

claude Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Reviewed; no blockers found.

removeEntry deleted a record's blob files up front, before (and independent of)
the record removal committing. When the removal didn't actually land — an
expiration scan whose transaction is force-committed without the delete, or an
aborted/version-conflicted removal — the record survived while its blob files
were unlinked, an orphaned reference that wedges replication on ENOENT.

Defer the blob deletion until the removal's commit promise resolves (it resolves
truthy on commit, false on a conditional-version miss, and rejects on abort), so
the unlink only happens once the delete is durable. Falls back to immediate
deletion when the store's remove() is synchronous. Can only ever skip an unlink,
never lose data; a skipped unlink leaves a reclaimable file (swept by
cleanupOrphans).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@kriszyp
kriszyp force-pushed the kris/blob-unlink-recheck-1364 branch from f89d7a0 to 45cf04b Compare June 18, 2026 02:04
@kriszyp
kriszyp marked this pull request as ready for review June 18, 2026 02:56
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@kriszyp
kriszyp merged commit 5077d9c into main Jun 18, 2026
81 of 83 checks passed
@kriszyp
kriszyp deleted the kris/blob-unlink-recheck-1364 branch June 18, 2026 18:45
kriszyp added a commit that referenced this pull request Jun 19, 2026
The harper-bench host serves harper + harper-pro via a repo-scoped just-in-time
supervisor (one job at a time), not an org-level runner — org/enterprise runners
aren't routed jobs for these repos. Also merges current main to pick up the blob
fixes (#1353, #1364/#1369) the stale branch was missing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cache expiration scan orphans blob files (unlink on fixed deletionDelay timer, decoupled from RocksDB commit)

1 participant