Skip to content

Minor updates to BaseMetastoreTableOperations#390

Merged
rdblue merged 2 commits intoapache:masterfrom
rdblue:update-base-metastore-table-ops
Aug 22, 2019
Merged

Minor updates to BaseMetastoreTableOperations#390
rdblue merged 2 commits intoapache:masterfrom
rdblue:update-base-metastore-table-ops

Conversation

@rdblue
Copy link
Contributor

@rdblue rdblue commented Aug 14, 2019

This includes 2 updates to BaseMetastoreTableOperations:

  • Use overwrite to create metadata JSON files to avoid S3 negative caching
  • Add an optional predicate to determine if loading metadata should be retried after an exception

@rdblue rdblue force-pushed the update-base-metastore-table-ops branch from a404ade to 50d1a7d Compare August 20, 2019 19:57
@rdblue rdblue requested a review from danielcweeks August 21, 2019 21:39
Copy link
Contributor

@danielcweeks danielcweeks left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

@rdblue rdblue force-pushed the update-base-metastore-table-ops branch from 309c7cc to 6160792 Compare August 22, 2019 18:49
@rdblue rdblue merged commit 92b5b45 into apache:master Aug 22, 2019
jamesbornholt added a commit to jamesbornholt/iceberg-python that referenced this pull request Feb 17, 2026
The Java implementation switched to using overwrite for metadata writes
in 2019 (apache/iceberg#390) to avoid S3 negative caching. The Python
implementation was never updated to match. This also fixes metadata
writes to stores that don't support ListObjectsV2 (e.g. S3 Tables),
since the exists() check in create() uses ListObjectsV2.

See: https://github.com/apache/iceberg/blob/bd8d2895847bdce71b119f4cb7a83b2e1a58ea73/core/src/main/java/org/apache/iceberg/BaseMetastoreTableOperations.java#L159-L162
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants