Skip to content

Add parallel file deletion support for ExpireSnapshots cleanup #668

@slfan1989

Description

@slfan1989

Background

ExpireSnapshots may delete many files during cleanup, including data files, manifest files, manifest list files, and statistics files.

Currently, ExpireSnapshots already uses FileIO ::DeleteFiles for grouped deletion. However, the cleanup path itself still processes deletion groups sequentially. For large tables with many expired files, this may make snapshot expiration cleanup slower than necessary.

Proposal

Add parallel file deletion support inside the ExpireSnapshots cleanup strategy.

The initial implementation can be intentionally scoped and conservative:

  • Keep the public ExpireSnapshots API unchanged.
  • Use an internal default parallelism value.
  • Apply parallelism only inside the cleanup deletion path.
  • Preserve existing best-effort cleanup semantics.
  • Continue supporting custom DeleteWith(...) callbacks.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions