Skip to content

The Metastore lock timeout led to the loss of metadata overwrite data. #15446

@yl09099

Description

@yl09099

Apache Iceberg version

1.10.1 (latest release)

Query engine

Spark

Please describe the bug 🐞

  1. We have a third-party service to clean up snapshot versions. During the cleaning process, if there is a long waiting time after the first confirmation of the lock's survival as shown;
  2. During the waiting period, there were other normal submissions of batch 2026-02-08, and metadata_location was updated;
  3. When the waiting for the third-party service ended, the metadata_location of batch 2026-02-08 was overwritten using the file of batch 2026-02-07 metadata_location.
  4. This ultimately led to the data loss of batch 2026-02-08.
Image

Willingness to contribute

  • I can contribute a fix for this bug independently
  • I would be willing to contribute a fix for this bug with guidance from the Iceberg community
  • I cannot contribute a fix for this bug at this time

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions