Bug/update libcudf to handle arrow12 changes - #13794
Merged
rapids-bot[bot] merged 7 commits intoAug 2, 2023
Merged
Conversation
On application shutdown libcudf needs to call `EnsureS3Finalized` before exit. This is tricky since the arrow might not be compiled with S3 support enabled. So use dlopen to check for the symbol existence, and call when it does exist. Only do this in the S3 tests for now as we figure out a longer term solution.
robertmaynard
added
bug
github-actions
Bot
added
libcudf
Contributor
Author
|
apache/arrow#36974 is the upstream issue which covers why we need to use dlopen + dlsym to call the S3 finialize function. |
bdice
reviewed
Aug 1, 2023
davidwendt
requested changes
Aug 1, 2023
Contributor
|
spark-rapids-jni runs into with the current state of the PR as the next issue |
Co-authored-by: Bradley Dice <bdice@bradleydice.com>
Co-authored-by: Bradley Dice <bdice@bradleydice.com>
davidwendt
approved these changes
Aug 1, 2023
davidwendt
left a comment
davidwendt
left a comment
Contributor
There was a problem hiding this comment.
Approving C++ changes.
gerashegalov
approved these changes
Aug 1, 2023
gerashegalov
left a comment
gerashegalov
left a comment
Contributor
There was a problem hiding this comment.
LGTM, spark-rapids-jni build is working with this PR. Thanks!
bdice
approved these changes
Aug 2, 2023
Contributor
Author
|
/merge |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Contiuation of #13790 as more changes are needed to support Arrow 12 builds from source ( both static and shared ). This fixes issues when building against Arrow with S3 enabled, and corrects missing
acerotargets.NVIDIA/cudf-spark-jni#1306
Checklist