Skip to content

MetadataLocation parser rejects Java HadoopCatalog's v<n>.metadata.json naming #2533

@SreeramGarlapati

Description

@SreeramGarlapati

problem

MetadataLocation::from_str only accepts the <version>-<uuid>.metadata.json format. java's HadoopTableOperations writes metadata as v1.metadata.json, v2.metadata.json, etc. when iceberg-rust tries to commit on top of a table created by java's HadoopCatalog, parsing fails:

Error: Invalid metadata file name: v1.metadata.json

this blocks any commit operation on java HadoopCatalog-created tables.

reproduction

  1. create a table using java iceberg with HadoopCatalog (produces v1.metadata.json)
  2. load the table in iceberg-rust via any catalog that stores the full metadata path
  3. attempt a Transaction::commit — fails at MetadataLocation::from_str

relevant code

crates/iceberg/src/catalog/metadata_location.rsparse_file_name splits on - which doesn't exist in the java format.

expected behavior

parse_file_name should accept both formats:

  • rust-native: <version>-<uuid>.metadata.json
  • java hadoop: v<n>.metadata.json

the version number is the only thing needed from the parse — it drives with_next_version().

notes

  • the iceberg spec does not prescribe a metadata file naming convention
  • java's HadoopCatalog is widely used in production (hadoop/HDFS deployments)
  • this is read+commit interop, not a full HadoopCatalog implementation

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions