Update to Iceberg 1.1.0 #15079
Merged
Update to Iceberg 1.1.0 #15079
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Fokko
commented
Nov 17, 2022
Fokko
commented
Nov 17, 2022
| try (Closeable ignored = writer) { | ||
| writer.delete(dataFilePath, 0, GenericRecord.create(icebergTable.schema())); |
There was a problem hiding this comment.
.delete has been deprecated for a while, and removed in 1.1.0
electrum
reviewed
Nov 17, 2022
plugin/trino-iceberg/src/test/java/io/trino/plugin/iceberg/TestIcebergV2.java
Outdated
Show resolved
Hide resolved
Fokko
force-pushed
the
fd-iceberg-1-1-0
branch
from
13e02c9
to
7509aff
Compare
Fokko
changed the title
Fokko
force-pushed
the
fd-iceberg-1-1-0
branch
2 times, most recently
from
a4940e3
to
ac1b635
Compare
findepi
reviewed
Nov 28, 2022
...n/trino-iceberg/src/test/java/io/trino/plugin/iceberg/TestIcebergMetadataFileOperations.java
Show resolved
Hide resolved
findepi
reviewed
Nov 28, 2022
|
CI result may be related |
|
I've opened apache/iceberg#6322 to fix the NPE, which is unfortunate that we didn't see it earlier. |
nastra
force-pushed
the
fd-iceberg-1-1-0
branch
3 times, most recently
from
e501033
to
c8ebb3c
Compare
findepi
reviewed
Nov 30, 2022
nastra
force-pushed
the
fd-iceberg-1-1-0
branch
from
c8ebb3c
to
3d992cf
Compare
This was referenced Dec 6, 2022
dongjoon-hyun
pushed a commit
to apache/orc
that referenced
this pull request
Dec 6, 2022
### Why are the changes needed? We're seeing some conflicts in the proto files when updating Iceberg in Trino: trinodb/trino#15079 (comment) I think we should exclude the proto files from the `nohive` jar. Before the change: ``` ➜ java git:(main) ✗ rm -rf core/target/ ➜ java git:(main) ✗ mvn package -DskipTests ➜ java git:(main) ✗ cd core/target ➜ target git:(main) ✗ unzip orc-core-1.9.0-SNAPSHOT-nohive.jar ➜ target git:(main) find . | grep -i "google/protobuf/" ./google/protobuf/timestamp.proto ./google/protobuf/field_mask.proto ./google/protobuf/api.proto ./google/protobuf/duration.proto ./google/protobuf/struct.proto ./google/protobuf/wrappers.proto ./google/protobuf/source_context.proto ./google/protobuf/any.proto ./google/protobuf/type.proto ./google/protobuf/empty.proto ./google/protobuf/compiler ./google/protobuf/compiler/plugin.proto ./google/protobuf/descriptor.proto ``` After the change: ``` ➜ java git:(main) ✗ rm -rf core/target/ ➜ java git:(main) ✗ mvn package -DskipTests ➜ java git:(main) ✗ cd core/target ➜ target git:(main) ✗ unzip orc-core-1.9.0-SNAPSHOT-nohive.jar ➜ target git:(main) ✗ find . | grep -i "google/protobuf/" ∅ ``` Interestingly enough, the normal jar doesn't contain these proto files: ``` ➜ java git:(main) ✗ rm -rf core/target/ ➜ java git:(main) ✗ mvn package -DskipTests ➜ java git:(main) ✗ cd core/target ➜ target git:(main) unzip orc-core-1.9.0-SNAPSHOT.jar| grep -i proto ➜ target git:(main) find . | grep -i "google/protobuf/" ∅ ``` ### What changes were proposed in this pull request? Exclude the proto files ### How was this patch tested? Inspecting the produced jar Closes #1333 Closes #1334 from Fokko/fd-exclude-proto-files. Authored-by: Fokko Driesprong <fokko@tabular.io> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
dongjoon-hyun
pushed a commit
to apache/orc
that referenced
this pull request
Dec 6, 2022
### Why are the changes needed? We're seeing some conflicts in the proto files when updating Iceberg in Trino: trinodb/trino#15079 (comment) I think we should exclude the proto files from the `nohive` jar. Before the change: ``` ➜ java git:(main) ✗ rm -rf core/target/ ➜ java git:(main) ✗ mvn package -DskipTests ➜ java git:(main) ✗ cd core/target ➜ target git:(main) ✗ unzip orc-core-1.9.0-SNAPSHOT-nohive.jar ➜ target git:(main) find . | grep -i "google/protobuf/" ./google/protobuf/timestamp.proto ./google/protobuf/field_mask.proto ./google/protobuf/api.proto ./google/protobuf/duration.proto ./google/protobuf/struct.proto ./google/protobuf/wrappers.proto ./google/protobuf/source_context.proto ./google/protobuf/any.proto ./google/protobuf/type.proto ./google/protobuf/empty.proto ./google/protobuf/compiler ./google/protobuf/compiler/plugin.proto ./google/protobuf/descriptor.proto ``` After the change: ``` ➜ java git:(main) ✗ rm -rf core/target/ ➜ java git:(main) ✗ mvn package -DskipTests ➜ java git:(main) ✗ cd core/target ➜ target git:(main) ✗ unzip orc-core-1.9.0-SNAPSHOT-nohive.jar ➜ target git:(main) ✗ find . | grep -i "google/protobuf/" ∅ ``` Interestingly enough, the normal jar doesn't contain these proto files: ``` ➜ java git:(main) ✗ rm -rf core/target/ ➜ java git:(main) ✗ mvn package -DskipTests ➜ java git:(main) ✗ cd core/target ➜ target git:(main) unzip orc-core-1.9.0-SNAPSHOT.jar| grep -i proto ➜ target git:(main) find . | grep -i "google/protobuf/" ∅ ``` ### What changes were proposed in this pull request? Exclude the proto files ### How was this patch tested? Inspecting the produced jar Closes #1333 Closes #1334 from Fokko/fd-exclude-proto-files. Authored-by: Fokko Driesprong <fokko@tabular.io> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org> (cherry picked from commit f6fee07) Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
dongjoon-hyun
pushed a commit
to apache/orc
that referenced
this pull request
Dec 6, 2022
We're seeing some conflicts in the proto files when updating Iceberg in Trino: trinodb/trino#15079 (comment) I think we should exclude the proto files from the `nohive` jar. Before the change: ``` ➜ java git:(main) ✗ rm -rf core/target/ ➜ java git:(main) ✗ mvn package -DskipTests ➜ java git:(main) ✗ cd core/target ➜ target git:(main) ✗ unzip orc-core-1.9.0-SNAPSHOT-nohive.jar ➜ target git:(main) find . | grep -i "google/protobuf/" ./google/protobuf/timestamp.proto ./google/protobuf/field_mask.proto ./google/protobuf/api.proto ./google/protobuf/duration.proto ./google/protobuf/struct.proto ./google/protobuf/wrappers.proto ./google/protobuf/source_context.proto ./google/protobuf/any.proto ./google/protobuf/type.proto ./google/protobuf/empty.proto ./google/protobuf/compiler ./google/protobuf/compiler/plugin.proto ./google/protobuf/descriptor.proto ``` After the change: ``` ➜ java git:(main) ✗ rm -rf core/target/ ➜ java git:(main) ✗ mvn package -DskipTests ➜ java git:(main) ✗ cd core/target ➜ target git:(main) ✗ unzip orc-core-1.9.0-SNAPSHOT-nohive.jar ➜ target git:(main) ✗ find . | grep -i "google/protobuf/" ∅ ``` Interestingly enough, the normal jar doesn't contain these proto files: ``` ➜ java git:(main) ✗ rm -rf core/target/ ➜ java git:(main) ✗ mvn package -DskipTests ➜ java git:(main) ✗ cd core/target ➜ target git:(main) unzip orc-core-1.9.0-SNAPSHOT.jar| grep -i proto ➜ target git:(main) find . | grep -i "google/protobuf/" ∅ ``` Exclude the proto files Inspecting the produced jar Closes #1333 Closes #1334 from Fokko/fd-exclude-proto-files. Authored-by: Fokko Driesprong <fokko@tabular.io> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org> (cherry picked from commit f6fee07) Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
dongjoon-hyun
pushed a commit
to apache/orc
that referenced
this pull request
Dec 6, 2022
We're seeing some conflicts in the proto files when updating Iceberg in Trino: trinodb/trino#15079 (comment) I think we should exclude the proto files from the `nohive` jar. Before the change: ``` ➜ java git:(main) ✗ rm -rf core/target/ ➜ java git:(main) ✗ mvn package -DskipTests ➜ java git:(main) ✗ cd core/target ➜ target git:(main) ✗ unzip orc-core-1.9.0-SNAPSHOT-nohive.jar ➜ target git:(main) find . | grep -i "google/protobuf/" ./google/protobuf/timestamp.proto ./google/protobuf/field_mask.proto ./google/protobuf/api.proto ./google/protobuf/duration.proto ./google/protobuf/struct.proto ./google/protobuf/wrappers.proto ./google/protobuf/source_context.proto ./google/protobuf/any.proto ./google/protobuf/type.proto ./google/protobuf/empty.proto ./google/protobuf/compiler ./google/protobuf/compiler/plugin.proto ./google/protobuf/descriptor.proto ``` After the change: ``` ➜ java git:(main) ✗ rm -rf core/target/ ➜ java git:(main) ✗ mvn package -DskipTests ➜ java git:(main) ✗ cd core/target ➜ target git:(main) ✗ unzip orc-core-1.9.0-SNAPSHOT-nohive.jar ➜ target git:(main) ✗ find . | grep -i "google/protobuf/" ∅ ``` Interestingly enough, the normal jar doesn't contain these proto files: ``` ➜ java git:(main) ✗ rm -rf core/target/ ➜ java git:(main) ✗ mvn package -DskipTests ➜ java git:(main) ✗ cd core/target ➜ target git:(main) unzip orc-core-1.9.0-SNAPSHOT.jar| grep -i proto ➜ target git:(main) find . | grep -i "google/protobuf/" ∅ ``` Exclude the proto files Inspecting the produced jar Closes #1333 Closes #1334 from Fokko/fd-exclude-proto-files. Authored-by: Fokko Driesprong <fokko@tabular.io> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org> (cherry picked from commit f6fee07) Signed-off-by: Dongjoon Hyun <dongjoon@apache.org> (cherry picked from commit 5ea8c06) Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
Fokko
force-pushed
the
fd-iceberg-1-1-0
branch
from
3d992cf
to
256c5b1
Compare
|
Just rebased against latest master |
Fokko
force-pushed
the
fd-iceberg-1-1-0
branch
from
256c5b1
to
446ded5
Compare
findepi
reviewed
Dec 12, 2022
plugin/trino-iceberg/src/main/java/io/trino/plugin/iceberg/IcebergSplitSource.java
Outdated
Show resolved
Hide resolved
Fokko
force-pushed
the
fd-iceberg-1-1-0
branch
from
446ded5
to
4177e3b
Compare
findepi
approved these changes
Dec 12, 2022
|
@findepi all green :) |
findepi
added
the
no-release-notes
|
@Fokko i assume this provides no end-user visible changes, correct? |
|
Thanks for merging @findepi! No user-facing changes, so we should be good 👍🏻 |
This was referenced Dec 21, 2022
cxzl25
pushed a commit
to cxzl25/orc
that referenced
this pull request
Jan 11, 2024
### Why are the changes needed? We're seeing some conflicts in the proto files when updating Iceberg in Trino: trinodb/trino#15079 (comment) I think we should exclude the proto files from the `nohive` jar. Before the change: ``` ➜ java git:(main) ✗ rm -rf core/target/ ➜ java git:(main) ✗ mvn package -DskipTests ➜ java git:(main) ✗ cd core/target ➜ target git:(main) ✗ unzip orc-core-1.9.0-SNAPSHOT-nohive.jar ➜ target git:(main) find . | grep -i "google/protobuf/" ./google/protobuf/timestamp.proto ./google/protobuf/field_mask.proto ./google/protobuf/api.proto ./google/protobuf/duration.proto ./google/protobuf/struct.proto ./google/protobuf/wrappers.proto ./google/protobuf/source_context.proto ./google/protobuf/any.proto ./google/protobuf/type.proto ./google/protobuf/empty.proto ./google/protobuf/compiler ./google/protobuf/compiler/plugin.proto ./google/protobuf/descriptor.proto ``` After the change: ``` ➜ java git:(main) ✗ rm -rf core/target/ ➜ java git:(main) ✗ mvn package -DskipTests ➜ java git:(main) ✗ cd core/target ➜ target git:(main) ✗ unzip orc-core-1.9.0-SNAPSHOT-nohive.jar ➜ target git:(main) ✗ find . | grep -i "google/protobuf/" ∅ ``` Interestingly enough, the normal jar doesn't contain these proto files: ``` ➜ java git:(main) ✗ rm -rf core/target/ ➜ java git:(main) ✗ mvn package -DskipTests ➜ java git:(main) ✗ cd core/target ➜ target git:(main) unzip orc-core-1.9.0-SNAPSHOT.jar| grep -i proto ➜ target git:(main) find . | grep -i "google/protobuf/" ∅ ``` ### What changes were proposed in this pull request? Exclude the proto files ### How was this patch tested? Inspecting the produced jar Closes apache#1333 Closes apache#1334 from Fokko/fd-exclude-proto-files. Authored-by: Fokko Driesprong <fokko@tabular.io> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
cxzl25
pushed a commit
to cxzl25/orc
that referenced
this pull request
Jan 11, 2024
We're seeing some conflicts in the proto files when updating Iceberg in Trino: trinodb/trino#15079 (comment) I think we should exclude the proto files from the `nohive` jar. Before the change: ``` ➜ java git:(main) ✗ rm -rf core/target/ ➜ java git:(main) ✗ mvn package -DskipTests ➜ java git:(main) ✗ cd core/target ➜ target git:(main) ✗ unzip orc-core-1.9.0-SNAPSHOT-nohive.jar ➜ target git:(main) find . | grep -i "google/protobuf/" ./google/protobuf/timestamp.proto ./google/protobuf/field_mask.proto ./google/protobuf/api.proto ./google/protobuf/duration.proto ./google/protobuf/struct.proto ./google/protobuf/wrappers.proto ./google/protobuf/source_context.proto ./google/protobuf/any.proto ./google/protobuf/type.proto ./google/protobuf/empty.proto ./google/protobuf/compiler ./google/protobuf/compiler/plugin.proto ./google/protobuf/descriptor.proto ``` After the change: ``` ➜ java git:(main) ✗ rm -rf core/target/ ➜ java git:(main) ✗ mvn package -DskipTests ➜ java git:(main) ✗ cd core/target ➜ target git:(main) ✗ unzip orc-core-1.9.0-SNAPSHOT-nohive.jar ➜ target git:(main) ✗ find . | grep -i "google/protobuf/" ∅ ``` Interestingly enough, the normal jar doesn't contain these proto files: ``` ➜ java git:(main) ✗ rm -rf core/target/ ➜ java git:(main) ✗ mvn package -DskipTests ➜ java git:(main) ✗ cd core/target ➜ target git:(main) unzip orc-core-1.9.0-SNAPSHOT.jar| grep -i proto ➜ target git:(main) find . | grep -i "google/protobuf/" ∅ ``` Exclude the proto files Inspecting the produced jar Closes apache#1333 Closes apache#1334 from Fokko/fd-exclude-proto-files. Authored-by: Fokko Driesprong <fokko@tabular.io> Signed-off-by: Dongjoon Hyun <dongjoon@apache.org> (cherry picked from commit f6fee07) Signed-off-by: Dongjoon Hyun <dongjoon@apache.org>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Testing the new release of Iceberg 1.1.0 against the Trino tests. This shouldn't be merged
Non-technical explanation
Release notes
( ) This is not user-visible or docs only and no release notes are required.
( ) Release notes are required, please propose a release note for me.
( ) Release notes are required, with the following suggested text: