Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI][Java][Integration] Integration verification jobs are failing due to libarrow_cdata_jni.so not found #40716

Closed
raulcd opened this issue Mar 21, 2024 · 6 comments
Closed

[CI][Java][Integration] Integration verification jobs are failing due to libarrow_cdata_jni.so not found #40716

raulcd opened this issue Mar 21, 2024 · 6 comments
Assignees
@danepitkin
Labels
Component: Continuous Integration Component: Integration Component: Java Priority: Blocker Marks a blocker for the release Type: bug
Milestone
16.0.0

Comments

@raulcd
Copy link
Member

raulcd commented Mar 21, 2024

Describe the bug, including details regarding any error messages, version, and platform.

Verification integration jobs have been failing for a long time:
verify-rc-source-integration-linux-almalinux-8-amd64
verify-rc-source-integration-linux-conda-latest-amd64
verify-rc-source-integration-linux-ubuntu-20.04-amd64
verify-rc-source-integration-linux-ubuntu-22.04-amd64
verify-rc-source-integration-macos-amd64
verify-rc-source-integration-macos-conda-amd64

It seems that they have been failing since the 19th of January. The commits between the last success and the first failure are: 55afcf0...05b8f36
I suspect #39001 is the culprit.

Based on the logs:

 [ERROR]   StreamTest.getNextError:192 ? IllegalState error loading native libraries: java.io.FileNotFoundException: arrow_cdata_jni/x86_64/libarrow_cdata_jni.so
[ERROR]   StreamTest.getSchemaError:213 ? IllegalState error loading native libraries: java.io.FileNotFoundException: arrow_cdata_jni/x86_64/libarrow_cdata_jni.so
[ERROR]   StreamTest.roundtripDictionary:169->roundtrip:228 ? IllegalState error loading native libraries: java.io.FileNotFoundException: arrow_cdata_jni/x86_64/libarrow_cdata_jni.so
[ERROR]   StreamTest.roundtripStrings:131->roundtrip:257->roundtrip:228 ? IllegalState error loading native libraries: java.io.FileNotFoundException: arrow_cdata_jni/x86_64/libarrow_cdata_jni.so
[ERROR]   StreamTest.testRoundtripInts:94->roundtrip:257->roundtrip:228 ? IllegalState error loading native libraries: java.io.FileNotFoundException: arrow_cdata_jni/x86_64/libarrow_cdata_jni.so

Component(s)

Continuous Integration, Integration, Java
@raulcd raulcd added the Priority: Blocker Marks a blocker for the release label Mar 21, 2024
@raulcd
Copy link
Member Author

raulcd commented Mar 21, 2024

cc @jduo @danepitkin

@jduo
Copy link
Member

jduo commented Mar 21, 2024

Yes it looks related to changing the location of JNI libraries. JNI libraries now need to be in <library_name>//.

@danepitkin
Copy link
Member

The verification scripts are mostly passing on main and fully passing in an open PR #40514. Are the verification script tests here failing in a different environment?

@raulcd
Copy link
Member Author

raulcd commented Mar 21, 2024

Those are on main and those are verify integration not verify java. I think the verify java are good.

@danepitkin
Copy link
Member

Thank you for the clarification!

@kou kou mentioned this issue Mar 21, 2024
Merged
kou pushed a commit that referenced this issue Mar 28, 2024
@danepitkin
GH-40716: [Java][Integration] Fix test_package_java in verification s…
50ca7a7 
…cripts (#40724)

### Rationale for this change

JPMS changed the location of JNI libs in the dist dir.

### What changes are included in this PR?

* Update the dist path in the verification script

### Are these changes tested?

CI

### Are there any user-facing changes?

No
* GitHub Issue: #40716

Authored-by: Dane Pitkin <dane@voltrondata.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
@kou kou added this to the 16.0.0 milestone Mar 28, 2024
@kou
Copy link
Member

kou commented Mar 28, 2024

Issue resolved by pull request 40724
#40724

@kou kou closed this as completed Mar 28, 2024
tolleybot pushed a commit to tmct/arrow that referenced this issue May 2, 2024
@danepitkin @tolleybot
apacheGH-40716: [Java][Integration] Fix test_package_java in verifica…
c06bd3f 
…tion scripts (apache#40724)

### Rationale for this change

JPMS changed the location of JNI libs in the dist dir.

### What changes are included in this PR?

* Update the dist path in the verification script

### Are these changes tested?

CI

### Are there any user-facing changes?

No
* GitHub Issue: apache#40716

Authored-by: Dane Pitkin <dane@voltrondata.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
vibhatha pushed a commit to vibhatha/arrow that referenced this issue May 25, 2024
@danepitkin @vibhatha
apacheGH-40716: [Java][Integration] Fix test_package_java in verifica…
e516ee2 
…tion scripts (apache#40724)

### Rationale for this change

JPMS changed the location of JNI libs in the dist dir.

### What changes are included in this PR?

* Update the dist path in the verification script

### Are these changes tested?

CI

### Are there any user-facing changes?

No
* GitHub Issue: apache#40716

Authored-by: Dane Pitkin <dane@voltrondata.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@danepitkin danepitkin

Labels
Component: Continuous Integration Component: Integration Component: Java Priority: Blocker Marks a blocker for the release Type: bug
Projects
None yet
Milestone
16.0.0
Development

No branches or pull requests
4 participants
@kou @raulcd @jduo @danepitkin