Skip to content

fix(retention): bound metadata-only segment census to one entry #868

Description

@ScriptedAlchemy

Problem

At central 27b2d3cc8d58c03d331d7bfdb52519feae18d940, the metadata-only code-generation retention census calls store_holds_generation_segments, which iterates code-generation-segments-v1/ until it finds a valid content-addressed segment or exhausts the directory. A store containing arbitrarily many unrelated, malformed, or crash-debris entries therefore makes the Doctor/storage-report path perform unbounded filesystem work. The callback is checked between entries, but the synchronous Doctor/storage wrapper supplies no cancellation.

The earlier metadata-only fix correctly stopped hashing multi-GiB manifests, but its claimed “bounded directory listing” is not actually bounded. Its contract only needs to distinguish a truly empty directory from one that may hold segment work: any observed entry, including debris, must conservatively retain/return typed Unknown. Exact filename classification is unnecessary here and would require unsafe filesystem cursor machinery.

The adjacent Unix non-UTF-8 filesystem capability probe also treats every write error as “unsupported,” so real failures such as ENOSPC/EACCES can silently skip coverage and emit a captured println! workaround.

Impact

A poisoned or debris-heavy profile can still stall operational diagnostics while enumerating segment metadata, exactly where diagnostics need to remain responsive. The silent probe skip can hide real CI/storage failures.

Acceptance

  • An empty or absent segment directory proves NoneFound.
  • Inspect at most one directory entry; any observed entry returns typed Unknown/held without filename classification.
  • Observe cancellation before accepting the observed entry.
  • Keep full-verification deletion fail-closed and unchanged.
  • Add deterministic large-debris behavioral coverage proving the census does not exhaust the directory.
  • The Unix non-UTF-8 probe skips only the platform/filesystem error that specifically means invalid byte names are unsupported; ENOSPC/EACCES/other write failures fail the test.
  • Remove the captured-output println! skip workaround.
  • No new cursor format, source-shape assertions, or budget increases.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions