Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[#2543] feat(spark-connector): support row-level operations to iceberg Table #3366

Commits on May 13, 2024

  1. [#2543] feat(spark-connector): support row-level operations to iceber…

    …g Table (#3243)
    
    ### What changes were proposed in this pull request?
    
    - refactor table implementation, make `SparkIcebergTable` extend Iceberg
    `SparkTable`, and `SparkHiveTable` extend Kyuubi `HiveTable`.
    
    - support row-level operations to iceberg Table
    
    ```
    1. update tableName set c1=v1, c2=v2, ...
    
    2. merge into targetTable t
       using sourceTable s
       on s.key=t.key
       when matched then ...
       when not matched then ...
    
    3. delete from table where xxx
    ```
    
    ### Why are the changes needed?
    
    1. For spark-connector in Iceberg, it explicitly uses `SparkTable` to
    identify whether it is an Iceberg table, so the `SparkIcebergTable` must
    extend `SparkTable`.
    
    2. support row-level operations to iceberg Table.
    
    Fix: #2543
    
    ### Does this PR introduce any user-facing change?
    Yes, support update ... , merge into ..., delete from ...
    
    ### How was this patch tested?
    New ITs.
    caican00 authored and web-flow committed May 13, 2024
    Configuration menu
    Copy the full SHA
    e40c5da View commit details
    Browse the repository at this point in the history