Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] JNI nightly build failing after upgrade to Arrow 9.x #11521

Closed
jlowe opened this issue Aug 12, 2022 · 1 comment · Fixed by #11535
Closed

[BUG] JNI nightly build failing after upgrade to Arrow 9.x #11521

jlowe opened this issue Aug 12, 2022 · 1 comment · Fixed by #11535
Assignees
Labels
bug Something isn't working CMake CMake build issue Spark Functionality that helps Spark RAPIDS

Comments

@jlowe
Copy link
Member

jlowe commented Aug 12, 2022

Describe the bug
After #11507 the nightly cudf JNI and spark-rapids-jni builds have been failing, sample failure from the nightly cudf snapshot jar publishing build:

07:19:06       [exec] CMake Error at CMakeLists.txt:129 (add_library):
07:19:06       [exec]   Target "cudfjni" links to target "arrow::flatbuffers" but the target was
07:19:06       [exec]   not found.  Perhaps a find_package() call is missing for an IMPORTED
07:19:06       [exec]   target, or an ALIAS target is missing?
07:19:06       [exec] 
07:19:06       [exec] 
07:19:06       [exec] CMake Error at CMakeLists.txt:129 (add_library):
07:19:06       [exec]   Target "cudfjni" links to target "arrow::hadoop" but the target was not
07:19:06       [exec]   found.  Perhaps a find_package() call is missing for an IMPORTED target, or
07:19:06       [exec]   an ALIAS target is missing?
07:19:06       [exec] 
07:19:06       [exec] 
07:19:06       [exec] CMake Error at CMakeLists.txt:161 (add_library):
07:19:06       [exec]   Target "cudfjnistub" links to target "arrow::flatbuffers" but the target
07:19:06       [exec]   was not found.  Perhaps a find_package() call is missing for an IMPORTED
07:19:06       [exec]   target, or an ALIAS target is missing?
07:19:06       [exec] 
07:19:06       [exec] 
07:19:06       [exec] CMake Error at CMakeLists.txt:161 (add_library):
07:19:06       [exec]   Target "cudfjnistub" links to target "arrow::hadoop" but the target was not
07:19:06       [exec]   found.  Perhaps a find_package() call is missing for an IMPORTED target, or
07:19:06       [exec]   an ALIAS target is missing?
07:19:06       [exec] 
07:19:06       [exec] 
07:19:06       [exec] CMake Error at CMakeLists.txt:203 (add_library):
07:19:06       [exec]   Target "cufilejni" links to target "arrow::flatbuffers" but the target was
07:19:06       [exec]   not found.  Perhaps a find_package() call is missing for an IMPORTED
07:19:06       [exec]   target, or an ALIAS target is missing?
07:19:06       [exec] 
07:19:06       [exec] 
07:19:06       [exec] CMake Error at CMakeLists.txt:203 (add_library):
07:19:06       [exec]   Target "cufilejni" links to target "arrow::hadoop" but the target was not
07:19:06       [exec]   found.  Perhaps a find_package() call is missing for an IMPORTED target, or
07:19:06       [exec]   an ALIAS target is missing?
07:19:06       [exec] 
07:19:06       [exec] 
07:19:06       [exec] -- Generating done
07:19:06       [exec] CMake Generate step failed.  Build files cannot be regenerated correctly.

The JNI build does not explicitly depend on the things it is complaining about, these are transitive dependencies from depending on libcudf. Note that this is a build outside of the conda environment, with -DCUDF_USE_ARROW_STATIC_ON and -DBUILD_SHARED_LIBS=OFF. My guess is that something about the Arrow 9.x static configuration is causing cmake to get confused on how to find these transitive Arrow dependencies from the libcudf cpp/build area.

Steps/Code to reproduce bug
Follow the build instructions in the cudf Java CI README.

Expected behavior
Build should complete successfully.

Environment overview (please complete the following information)

  • Docker environment
  • Not building with Conda
  • Static Arrow libraries with a static libcudf library build
@jlowe jlowe added bug Something isn't working Needs Triage Need team to review and classify CMake CMake build issue Spark Functionality that helps Spark RAPIDS labels Aug 12, 2022
@github-actions github-actions bot added this to Needs prioritizing in Bug Squashing Aug 12, 2022
@robertmaynard robertmaynard self-assigned this Aug 12, 2022
@revans2
Copy link
Contributor

revans2 commented Aug 15, 2022

Because of NVIDIA/spark-rapids-jni#479 I think this is now blocking a lot of work on spark-rapids.

robertmaynard added a commit to robertmaynard/cudf that referenced this issue Aug 15, 2022
arrow 9's CMake code generates new imported interface targets
which cudf needs to replicate so that consumers of cudf don't
get errors abount `arrow::hadoop` or `arrow::flatbuffers`.

Fixes rapidsai#11521
rapids-bot bot pushed a commit that referenced this issue Aug 15, 2022
arrow 9's CMake code generates new imported interface targets which cudf needs to replicate so that consumers of cudf don't get errors abount `arrow::hadoop` or `arrow::flatbuffers`.

Fixes #11521

Authors:
  - Robert Maynard (https://github.com/robertmaynard)

Approvers:
  - Jason Lowe (https://github.com/jlowe)
  - Bradley Dice (https://github.com/bdice)

URL: #11535
Bug Squashing automation moved this from Needs prioritizing to Closed Aug 15, 2022
@bdice bdice removed the Needs Triage Need team to review and classify label Mar 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working CMake CMake build issue Spark Functionality that helps Spark RAPIDS
Projects
No open projects
Development

Successfully merging a pull request may close this issue.

4 participants