[#2543] feat(spark-connector): support row-level operations to iceberg Table #3366
Conversation
…g Table (#3243) ### What changes were proposed in this pull request? - refactor table implementation, make `SparkIcebergTable` extend Iceberg `SparkTable`, and `SparkHiveTable` extend Kyuubi `HiveTable`. - support row-level operations to iceberg Table ``` 1. update tableName set c1=v1, c2=v2, ... 2. merge into targetTable t using sourceTable s on s.key=t.key when matched then ... when not matched then ... 3. delete from table where xxx ``` ### Why are the changes needed? 1. For spark-connector in Iceberg, it explicitly uses `SparkTable` to identify whether it is an Iceberg table, so the `SparkIcebergTable` must extend `SparkTable`. 2. support row-level operations to iceberg Table. Fix: #2543 ### Does this PR introduce any user-facing change? Yes, support update ... , merge into ..., delete from ... ### How was this patch tested? New ITs.
|
@caican00 could analyse the erorr and fix it? |
|
this PR depends on #3358 |
|
@caican00 could you fix the conflict? |
|
Why there is a conflict in this part? AFAIK this part should be quite close in branch-0.5 and main. |
Maybe a github bug, I could patch the PR to branch-0.5 manually without modification. |
|
Anyway, @caican00 would you please cherry-pick this commit to branch-0.5 and create a new PR? I'm going to close this PR. |
Sorry for the late reply. i have created a new pr for this commit to branch-0.5 cc @jerryshao @FANNG1 |
jerryshao
deleted the
cherry-pick-branch-0.5-f207466f143dc21cb4fd015e7b98c1ab26782025
branch
What changes were proposed in this pull request?
refactor table implementation, make
SparkIcebergTableextend IcebergSparkTable, andSparkHiveTableextend KyuubiHiveTable.support row-level operations to iceberg Table
Why are the changes needed?
For spark-connector in Iceberg, it explicitly uses
SparkTableto identify whether it is an Iceberg table, so theSparkIcebergTablemust extendSparkTable.support row-level operations to iceberg Table.
Fix: #2543
Does this PR introduce any user-facing change?
Yes, support update ... , merge into ..., delete from ...
How was this patch tested?
New ITs.