Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[C++] Installed plasma-store-server fails finding Boost #24067

Closed
asfimport opened this issue Feb 12, 2020 · 8 comments
Closed

[C++] Installed plasma-store-server fails finding Boost #24067

asfimport opened this issue Feb 12, 2020 · 8 comments
Assignees
@pitrou
Labels
Component: C++ - Plasma Component: C++ Type: bug
Milestone
0.17.0

Comments

@asfimport
Copy link
Collaborator

In my build directory I have:

$ ldd build-test/debug/plasma-store-server 
	linux-vdso.so.1 (0x00007ffc0001f000)
	libplasma.so.100 => /home/antoine/arrow/dev/cpp/build-test/debug/libplasma.so.100 (0x00007efbff629000)
	libarrow_cuda.so.100 => /home/antoine/arrow/dev/cpp/build-test/debug/libarrow_cuda.so.100 (0x00007efbff58d000)
	libarrow.so.100 => /home/antoine/arrow/dev/cpp/build-test/debug/libarrow.so.100 (0x00007efbfcbae000)
	libssl.so.1.1 => /home/antoine/miniconda3/envs/pyarrow/lib/libssl.so.1.1 (0x00007efbfcb1e000)
	libcrypto.so.1.1 => /home/antoine/miniconda3/envs/pyarrow/lib/libcrypto.so.1.1 (0x00007efbfc870000)
	libaws-cpp-sdk-config.so => /home/antoine/miniconda3/envs/pyarrow/lib/libaws-cpp-sdk-config.so (0x00007efbfc6be000)
	libaws-cpp-sdk-transfer.so => /home/antoine/miniconda3/envs/pyarrow/lib/libaws-cpp-sdk-transfer.so (0x00007efbff557000)
	libaws-cpp-sdk-s3.so => /home/antoine/miniconda3/envs/pyarrow/lib/libaws-cpp-sdk-s3.so (0x00007efbfc478000)
	libaws-cpp-sdk-core.so => /home/antoine/miniconda3/envs/pyarrow/lib/libaws-cpp-sdk-core.so (0x00007efbfc37b000)
	libaws-c-event-stream.so.0unstable => /home/antoine/miniconda3/envs/pyarrow/lib/libaws-c-event-stream.so.0unstable (0x00007efbff54e000)
	libaws-c-common.so.0unstable => /home/antoine/miniconda3/envs/pyarrow/lib/libaws-c-common.so.0unstable (0x00007efbff52d000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007efbfbfa2000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007efbfbd83000)
	libaws-checksums.so => /home/antoine/miniconda3/envs/pyarrow/lib/libaws-checksums.so (0x00007efbff51d000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007efbfbb7b000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007efbfb977000)
	libcuda.so.1 => /usr/lib/x86_64-linux-gnu/libcuda.so.1 (0x00007efbfadd7000)
	libstdc++.so.6 => /home/antoine/miniconda3/envs/pyarrow/lib/libstdc++.so.6 (0x00007efbfac63000)
	libgcc_s.so.1 => /home/antoine/miniconda3/envs/pyarrow/lib/libgcc_s.so.1 (0x00007efbff507000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007efbfa872000)
	/lib64/ld-linux-x86-64.so.2 (0x00007efbff4d7000)
	libbz2.so.1.0 => /home/antoine/miniconda3/envs/pyarrow/lib/libbz2.so.1.0 (0x00007efbfa85e000)
	liblz4.so.1 => /home/antoine/miniconda3/envs/pyarrow/lib/liblz4.so.1 (0x00007efbfa829000)
	libsnappy.so.1 => /home/antoine/miniconda3/envs/pyarrow/lib/libsnappy.so.1 (0x00007efbfa81e000)
	libz.so.1 => /home/antoine/miniconda3/envs/pyarrow/lib/libz.so.1 (0x00007efbfa804000)
	libzstd.so.1 => /home/antoine/miniconda3/envs/pyarrow/lib/libzstd.so.1 (0x00007efbfa748000)
	libboost_filesystem.so.1.68.0 => /home/antoine/miniconda3/envs/pyarrow/lib/libboost_filesystem.so.1.68.0 (0x00007efbfa72a000)
	libboost_system.so.1.68.0 => /home/antoine/miniconda3/envs/pyarrow/lib/libboost_system.so.1.68.0 (0x00007efbff4fe000)
	libcurl.so.4 => /home/antoine/miniconda3/envs/pyarrow/lib/./libcurl.so.4 (0x00007efbfa6a4000)
	libnvidia-fatbinaryloader.so.390.116 => /usr/lib/x86_64-linux-gnu/libnvidia-fatbinaryloader.so.390.116 (0x00007efbfa456000)
	libssh2.so.1 => /home/antoine/miniconda3/envs/pyarrow/lib/././libssh2.so.1 (0x00007efbfa423000)
	libgssapi_krb5.so.2 => /home/antoine/miniconda3/envs/pyarrow/lib/././libgssapi_krb5.so.2 (0x00007efbfa3d4000)
	libkrb5.so.3 => /home/antoine/miniconda3/envs/pyarrow/lib/././libkrb5.so.3 (0x00007efbfa2fd000)
	libk5crypto.so.3 => /home/antoine/miniconda3/envs/pyarrow/lib/././libk5crypto.so.3 (0x00007efbfa2de000)
	libcom_err.so.3 => /home/antoine/miniconda3/envs/pyarrow/lib/././libcom_err.so.3 (0x00007efbfa2d6000)
	libkrb5support.so.0 => /home/antoine/miniconda3/envs/pyarrow/lib/./././libkrb5support.so.0 (0x00007efbfa2c8000)
	libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007efbfa0ad000)

However, once installed it seems the Boost resolution fails:

$ ldd /home/antoine/miniconda3/envs/pyarrow/bin/plasma-store-server
	linux-vdso.so.1 (0x00007ffc0001f000)
	libplasma.so.100 => /home/antoine/miniconda3/envs/pyarrow/lib/libplasma.so.100 (0x00007efbff629000)
	libarrow_cuda.so.100 => /home/antoine/miniconda3/envs/pyarrow/lib/libarrow_cuda.so.100 (0x00007efbff58d000)
	libarrow.so.100 => /home/antoine/miniconda3/envs/pyarrow/lib/libarrow.so.100 (0x00007efbfcbae000)
	libssl.so.1.1 => /home/antoine/miniconda3/envs/pyarrow/lib/libssl.so.1.1 (0x00007efbfcb1e000)
	libcrypto.so.1.1 => /home/antoine/miniconda3/envs/pyarrow/lib/libcrypto.so.1.1 (0x00007efbfc870000)
	libaws-cpp-sdk-config.so => /home/antoine/miniconda3/envs/pyarrow/lib/libaws-cpp-sdk-config.so (0x00007efbfc6be000)
	libaws-cpp-sdk-transfer.so => /home/antoine/miniconda3/envs/pyarrow/lib/libaws-cpp-sdk-transfer.so (0x00007efbff557000)
	libaws-cpp-sdk-s3.so => /home/antoine/miniconda3/envs/pyarrow/lib/libaws-cpp-sdk-s3.so (0x00007efbfc478000)
	libaws-cpp-sdk-core.so => /home/antoine/miniconda3/envs/pyarrow/lib/libaws-cpp-sdk-core.so (0x00007efbfc37b000)
	libaws-c-event-stream.so.0unstable => /home/antoine/miniconda3/envs/pyarrow/lib/libaws-c-event-stream.so.0unstable (0x00007efbff54e000)
	libaws-c-common.so.0unstable => /home/antoine/miniconda3/envs/pyarrow/lib/libaws-c-common.so.0unstable (0x00007efbff52d000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007efbfbfa2000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007efbfbd83000)
	libaws-checksums.so => /home/antoine/miniconda3/envs/pyarrow/lib/libaws-checksums.so (0x00007efbff51d000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007efbfbb7b000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007efbfb977000)
	libcuda.so.1 => /usr/lib/x86_64-linux-gnu/libcuda.so.1 (0x00007efbfadd7000)
	libstdc++.so.6 => /home/antoine/miniconda3/envs/pyarrow/lib/libstdc++.so.6 (0x00007efbfac63000)
	libgcc_s.so.1 => /home/antoine/miniconda3/envs/pyarrow/lib/libgcc_s.so.1 (0x00007efbff507000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007efbfa872000)
	/lib64/ld-linux-x86-64.so.2 (0x00007efbff4d7000)
	libbz2.so.1.0 => /lib/x86_64-linux-gnu/libbz2.so.1.0 (0x00007efbfa662000)
	liblz4.so.1 => /usr/lib/x86_64-linux-gnu/liblz4.so.1 (0x00007efbfa446000)
	libsnappy.so.1 => /usr/lib/x86_64-linux-gnu/libsnappy.so.1 (0x00007efbfa23e000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007efbfa021000)
	libzstd.so.1 => /usr/lib/x86_64-linux-gnu/libzstd.so.1 (0x00007efbf9da6000)
	libboost_filesystem.so.1.68.0 => not found
	libboost_system.so.1.68.0 => not found
	libcurl.so.4 => /home/antoine/miniconda3/envs/pyarrow/lib/./libcurl.so.4 (0x00007efbf9d20000)
	libnvidia-fatbinaryloader.so.390.116 => /usr/lib/x86_64-linux-gnu/libnvidia-fatbinaryloader.so.390.116 (0x00007efbf9ad4000)
	libssh2.so.1 => /home/antoine/miniconda3/envs/pyarrow/lib/././libssh2.so.1 (0x00007efbf9aa1000)
	libgssapi_krb5.so.2 => /home/antoine/miniconda3/envs/pyarrow/lib/././libgssapi_krb5.so.2 (0x00007efbf9a52000)
	libkrb5.so.3 => /home/antoine/miniconda3/envs/pyarrow/lib/././libkrb5.so.3 (0x00007efbf997b000)
	libk5crypto.so.3 => /home/antoine/miniconda3/envs/pyarrow/lib/././libk5crypto.so.3 (0x00007efbf995c000)
	libcom_err.so.3 => /home/antoine/miniconda3/envs/pyarrow/lib/././libcom_err.so.3 (0x00007efbf9956000)
	libkrb5support.so.0 => /home/antoine/miniconda3/envs/pyarrow/lib/./././libkrb5support.so.0 (0x00007efbf9948000)
	libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007efbf972d000)

Other libraries in the same directory are found, so I don't know what's happening here.

Reporter: Antoine Pitrou / @pitrou
Assignee: Antoine Pitrou / @pitrou

PRs and other links:

Note: This issue was originally created as ARROW-7838. Please see the migration documentation for further details.
@asfimport
Copy link
Collaborator Author

Antoine Pitrou / @pitrou:
cc @kou would you have a hint here?

@asfimport
Copy link
Collaborator Author

Antoine Pitrou / @pitrou:
If I enable static linking (ARROW_BUILD_STATIC), then apparently it works fine:

$ ldd /home/antoine/miniconda3/envs/pyarrow/bin/plasma-store-server
	linux-vdso.so.1 (0x00007ffc0001f000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007efbff2d3000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007efbff0cb000)
	libcuda.so.1 => /usr/lib/x86_64-linux-gnu/libcuda.so.1 (0x00007efbfe52b000)
	libssl.so.1.1 => /home/antoine/miniconda3/envs/pyarrow/lib/libssl.so.1.1 (0x00007efbff631000)
	libcrypto.so.1.1 => /home/antoine/miniconda3/envs/pyarrow/lib/libcrypto.so.1.1 (0x00007efbfe27d000)
	libbz2.so.1.0 => /home/antoine/miniconda3/envs/pyarrow/lib/libbz2.so.1.0 (0x00007efbff61d000)
	liblz4.so.1 => /home/antoine/miniconda3/envs/pyarrow/lib/liblz4.so.1 (0x00007efbff5e6000)
	libsnappy.so.1 => /home/antoine/miniconda3/envs/pyarrow/lib/libsnappy.so.1 (0x00007efbff5db000)
	libz.so.1 => /home/antoine/miniconda3/envs/pyarrow/lib/libz.so.1 (0x00007efbff5c1000)
	libzstd.so.1 => /home/antoine/miniconda3/envs/pyarrow/lib/libzstd.so.1 (0x00007efbff505000)
	libboost_filesystem.so.1.68.0 => /home/antoine/miniconda3/envs/pyarrow/lib/libboost_filesystem.so.1.68.0 (0x00007efbfe25f000)
	libboost_system.so.1.68.0 => /home/antoine/miniconda3/envs/pyarrow/lib/libboost_system.so.1.68.0 (0x00007efbff4fe000)
	libstdc++.so.6 => /home/antoine/miniconda3/envs/pyarrow/lib/libstdc++.so.6 (0x00007efbfe0e9000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007efbfdd4b000)
	libgcc_s.so.1 => /home/antoine/miniconda3/envs/pyarrow/lib/libgcc_s.so.1 (0x00007efbfdd37000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007efbfdb18000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007efbfd727000)
	/lib64/ld-linux-x86-64.so.2 (0x00007efbff4d7000)
	libnvidia-fatbinaryloader.so.390.116 => /usr/lib/x86_64-linux-gnu/libnvidia-fatbinaryloader.so.390.116 (0x00007efbfd4db000)

But static linking is wasteful, it would be good if we could avoid it.

@asfimport
Copy link
Collaborator Author

Kouhei Sutou / @kou:
What CMake options do you use?
I couldn't build plasma-store-server that links to libboost* with my CMake options.

@asfimport
Copy link
Collaborator Author

Antoine Pitrou / @pitrou:
Ah, apparently you need ARROW_S3 enabled to have Boost linked in. I'm not sure why...

@asfimport
Copy link
Collaborator Author

Antoine Pitrou / @pitrou:
Hmm, so the S3 and Flight tests need Boost. But Boost shouldn't be linked with libarrow.so.

@asfimport
Copy link
Collaborator Author

Kouhei Sutou / @kou:
Issue resolved by pull request 6438
#6438

@asfimport
Copy link
Collaborator Author

Wes McKinney / @wesm:
I tagged this with 0.16.1 since it probably all Boost-related patches would need to be included in a patch release

@asfimport
Copy link
Collaborator Author

Antoine Pitrou / @pitrou:
From the users' side, they're probably more improvements than bug fixes.

You need to build tests to trigger this one. The other one is simply an optimization where we don't bundle unneeded DLLs.

@asfimport asfimport added this to the 0.17.0 milestone Jan 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pitrou pitrou

Labels
Component: C++ - Plasma Component: C++ Type: bug
Projects
None yet
Milestone
0.17.0
Development

No branches or pull requests
2 participants
@pitrou @asfimport