-
Notifications
You must be signed in to change notification settings - Fork 434
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[GLUTEN-4933][VL] Update iceberg version to 1.4.3 for Spark 3.4 and above #4967
Conversation
Thanks for opening a pull request! Could you open an issue for this pull request on Github Issues? https://github.com/apache/incubator-gluten/issues Then could you also rename commit message and pull request title in the following format?
See also: |
Run Gluten Clickhouse CI |
related: #4933 |
@liujiayi771 UT shows incompatible |
@yma11 can we use different iceberg for different spark? sounds this would fix the issue |
@yma11 I think we should keep iceberg 1.3.1 for Spark 3.3. This compatibility issue was discovered before, which is why the Iceberg version 1.3.1 was chosen at that time. |
Run Gluten Clickhouse CI |
Run Gluten Clickhouse CI |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍
What changes were proposed in this pull request?
Update iceberg version to 1.4.3 for Spark 3.3 and above, as v1.3.1 is not compatible with Spark 3.5.
Note that iceberg's latest version is 1.5.0 but as it's not available in mirror repo, we use 1.4.3 instead. So the version mapping is:
How was this patch tested?
Existing UTs