Skip to content

[EPIC] Implement Missing Write Actions #2269

@Shekharrajak

Description

@Shekharrajak

iceberg-rust currently supports FastAppendAction for simple appends but is missing several transaction actions available in the Java Iceberg API that are needed for complete write support.

Java API (api/src/main/java/org/apache/iceberg/) iceberg-rust (crates/iceberg/src/transaction/) Status
AppendFiles.java append.rs (FastAppendAction) Available
OverwriteFiles.java - Missing
ReplacePartitions.java - Missing
DeleteFiles.java - Missing
RowDelta.java - Missing
RewriteFiles.java - Missing

TODOs:

  • Implement OverwriteAction - needed for INSERT OVERWRITE (static partition overwrite)
  • Implement ReplacePartitionsAction - needed for dynamic partition overwrite
  • Implement DeleteFilesAction - needed for DELETE FROM table operations
  • Implement RowDeltaAction - needed for row-level DELETE, UPDATE, MERGE INTO (CoW/MoR)
  • Implement RewriteFilesAction - needed for compaction and file rewriting

Related open issues:

#1104 - Support RowDeltaAction
#2202 - Implement RowDeltaAction for row-level modifications (CoW)
#2201 - Add MERGE INTO support for DataFusion integration
#2205 - SQL UPDATE support for DataFusion integration
#2186 - Copy-on-Write and Merge-on-Read support
#1382 - [EPIC] Support for appending data to iceberg table
#2218 - Support for a DeltaWriter

Motivation:

These transaction actions are required for apache/datafusion-comet to implement a fully native Iceberg write path (datafusion-comet#3595, datafusion-comet#2967). Comet currently writes Parquet data files natively via Rust/DataFusion but still relies on the Java Iceberg layer for table commits. Completing these actions in iceberg-rust would enable end-to-end native Iceberg writes - from Arrow batches through DataFusion execution to Iceberg table commits - eliminating JVM dependency for the entire write path.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions