Skip to content

[Bug] Snapshot expiration won't work, space usage and file number continue to grow #4743

@bridgeDream

Description

@bridgeDream

Search before asking

  • I searched in the issues and found nothing similar.

Paimon version

0.9

Compute Engine

flink 1.16

Minimal reproduce step

  1. starting a flink job writing to paimon with snapshot.time-retained configured.
  2. delete some snapshot (>10) between earliest and latest manually
  3. after a while, flink job while throw exception frequently like as
    image
  4. Snapshot expiration won't work anymore, Space usage and the number of files will continue to grow

What doesn't meet your expectations?

Even if the snapshot sequence is incomplete, Paimon should have a certain level of robustness to complete the snapshot expiration process rather than simply stop functioning.

Anything else?

No response

Are you willing to submit a PR?

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions