You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Today Iceberg writes only support merge-on-read mode. Copy-on-write mode is a frequent ask for users that want better file layout without the need to run compactions frequently.
Technically this could be achieved pretty easily. The CoW implementation is already available for Delta:
Yes, I think we should support copy-on-write mode for UPDATE and DELETE for case when we update/delete most of the file. Creating big delta files isn't helpful for anyone.
If we support this strategy, we should automatically choose between copy-on-write and merge-on-read within each IcebergMergeSink independently.
@findepi, there's a table setting for choosing between copy-on-write and merge-on-read: write.(operation).mode where operation can be merge, update, or delete and mode can be copy-on-write or merge-on-read.
Today Iceberg writes only support merge-on-read mode. Copy-on-write mode is a frequent ask for users that want better file layout without the need to run compactions frequently.
Technically this could be achieved pretty easily. The CoW implementation is already available for Delta:
https://github.com/trinodb/trino/blob/master/plugin/trino-delta-lake/src/main/java/io/trino/plugin/deltalake/DeltaLakeMergeSink.java
The text was updated successfully, but these errors were encountered: