Skip to content

docs(lancedb): mark the husk sweep removable once lance#8322 ships - #399

Merged
gloryfromca merged 2 commits into
mainfrom
docs/husk-sweep-upstream-marker
Aug 7, 2026
Merged

docs(lancedb): mark the husk sweep removable once lance#8322 ships#399
gloryfromca merged 2 commits into
mainfrom
docs/husk-sweep-upstream-marker

Conversation

@gloryfromca

Copy link
Copy Markdown
Collaborator

Follow-up to #392. Upstream acted on one of the two issues that PR's investigation produced.

lance-format/lance#8320cleanup_old_versions unlinks a superseded index's files but never removes the directory — was fixed by lance#8322, merged 2026-08-06. It removes the emptied directories under the same 7-day / delete_unverified policy that _remove_empty_index_dirs reimplements on our side.

The fix is not in any release yet: lancedb 0.34.0 embeds lance 8.0.0, and the merge commit sits ahead of v11.0.0-beta.2. So the sweep stays for now. This records why it should not stay forever, next to the code rather than in a chat log.

Two consequences worth having written down at the point of use:

  • Delete, don't keep both. Once a pin carries #8322, _remove_empty_index_dirs and its call site should go — a second implementation of upstream's own rule is the kind of thing that survives for years because nobody remembers it was a workaround.
  • The documented horizon halves. The constant currently explains a ~14-day steady state, because lance frees the files at 7 days, which resets the directory's mtime, and only then does our 7-day gate start. Upstream drops the directory in the same pass that empties it, so the two windows stop stacking and it goes back to ~7 days (~890k dirs / ~3.6GB at ceiling load, from ~1.8M / ~7GB).

Docs only — no behaviour change. ruff clean, 1938 unit tests passing.

🤖 Generated with Claude Code

Upstream merged the fix on 2026-08-06: cleanup now removes the directories it
empties, under the same 7-day / delete_unverified policy this sweep
reimplements. It is not in any release yet (lancedb 0.34.0 embeds lance 8.0.0;
the merge sits ahead of v11.0.0-beta.2), so the sweep stays -- but the next pin
carrying it should delete this rather than keep a second copy of upstream's
rule. That upgrade also collapses the documented horizon from ~14 days to ~7:
lance drops the directory in the same pass that empties it, so our age gate
stops stacking on top of theirs.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@gloryfromca
gloryfromca merged commit 560fb80 into main Aug 7, 2026
9 checks passed
@gloryfromca
gloryfromca deleted the docs/husk-sweep-upstream-marker branch August 7, 2026 07:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants