Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[C++] libgandiva.so links with libzstd.so even though the Gandiva module doesn't depend on zstd and zstd isn't enabled #37410

Closed
yurivict opened this issue Aug 27, 2023 · 5 comments · Fixed by #37412
Assignees
Milestone

Comments

@yurivict
Copy link

Describe the bug, including details regarding any error messages, version, and platform.

Here are cmake args:

-DCMAKE_CXX_STANDARD=17 -DARROW_SIMD_LEVEL=NONE -DARROW_RUNTIME_SIMD_LEVEL=NONE -DPython3_EXECUTABLE=/usr/local/bin/python3.9 -DARROW_ACERO:BOOL=true -DARROW_WITH_BROTLI:BOOL=false -DARROW_BUILD_UTILITIES:BOOL=true -DARROW_WITH_BZ2:BOOL=false -DARROW_COMPUTE:BOOL=true -DARROW_CSV:BOOL=true -DARROW_DATASET:BOOL=true -DARROW_FILESYSTEM:BOOL=true -DARROW_FLIGHT:BOOL=false -DARROW_GANDIVA:BOOL=true -DARROW_HDFS:BOOL=true -DARROW_IPC:BOOL=true -DARROW_JSON:BOOL=true -DARROW_WITH_LZ4:BOOL=false -DARROW_MIMALLOC:BOOL=false -DARROW_ORC:BOOL=false -DARROW_PARQUET:BOOL=true -DPARQUET_REQUIRE_ENCRYPTION:BOOL=true -DARROW_S3:BOOL=true -DARROW_SKYHOOK:BOOL=false -DARROW_WITH_SNAPPY:BOOL=false -DARROW_SUBSTRAIT:BOOL=false -DARROW_TENSORFLOW:BOOL=false -DARROW_TESTING:BOOL=true -DARROW_WITH_UTF8PROC:BOOL=true -DARROW_WITH_ZLIB:BOOL=false -DARROW_WITH_ZSTD:BOOL=false -DCMAKE_C_COMPILER:STRING="cc" -DCMAKE_CXX_COMPILER:STRING="c++" -DCMAKE_C_FLAGS:STRING="-O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing " -DCMAKE_C_FLAGS_DEBUG:STRING="-O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing " -DCMAKE_C_FLAGS_RELEASE:STRING="-O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -DNDEBUG" -DCMAKE_CXX_FLAGS:STRING="-O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -isystem /usr/local/include " -DCMAKE_CXX_FLAGS_DEBUG:STRING="-O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -isystem /usr/local/include " -DCMAKE_CXX_FLAGS_RELEASE:STRING="-O2 -pipe -fstack-protector-strong -isystem /usr/local/include -fno-strict-aliasing -isystem /usr/local/include -DNDEBUG" -DCMAKE_EXE_LINKER_FLAGS:STRING=" -fstack-protector-strong " -DCMAKE_MODULE_LINKER_FLAGS:STRING=" -fstack-protector-strong " -DCMAKE_SHARED_LINKER_FLAGS:STRING=" -fstack-protector-strong " -DCMAKE_INSTALL_PREFIX:PATH="/usr/local" -DCMAKE_BUILD_TYPE:STRING="Release" -DTHREADS_HAVE_PTHREAD_ARG:BOOL=YES -DCMAKE_INSTALL_RPATH_USE_LINK_PATH=YES -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON -DFETCHCONTENT_FULLY_DISCONNECTED:BOOL=ON -DARROW_BUILD_STATIC:BOOL=OFF -DARROW_WITH_BACKTRACE:BOOL=OFF -DARROW_BUILD_TESTS:BOOL=OFF -DARROW_BUILD_BENCHMARKS:BOOL=OFF -GNinja -DPython_ADDITIONAL_VERSIONS=3.9 -DBOOST_PYTHON_SUFFIX:STRING=39

Version: 13.0.0
FreeBSD 13.2

Component(s)

C++

@yurivict yurivict changed the title libgandiva.so links with libzstd.so even though Gthe Gandiva module doesn't depend on zstd and zstd isn't enabled libgandiva.so links with libzstd.so even though the Gandiva module doesn't depend on zstd and zstd isn't enabled Aug 27, 2023
@kou kou changed the title libgandiva.so links with libzstd.so even though the Gandiva module doesn't depend on zstd and zstd isn't enabled [C++] libgandiva.so links with libzstd.so even though the Gandiva module doesn't depend on zstd and zstd isn't enabled Aug 27, 2023
@kou
Copy link
Member

kou commented Aug 27, 2023

How did you confirm that libgandiva.so links with libzstd.so?

@yurivict
Copy link
Author

The FreeBSD ports framework complains:

====> Running Q/A tests (stage-qa)
Error: /usr/local/lib/libgandiva.so.1300.0.0 is linked to /usr/local/lib/libzstd.so.1 from archivers/zstd but it is not declared as a dependency
Warning: you need LIB_DEPENDS+=libzstd.so:archivers/zstd

@kou
Copy link
Member

kou commented Aug 27, 2023

I think that it's caused by LLVM. LLVM may use libzstd.so. (I don't know whether FreeBSD's LLVM uses libzstd.so or not.)
If LLVM uses libzstd.so, libgandiva.so depends on libzstd.so implicitly.

@yurivict
Copy link
Author

Does Gandiva link statically with LLVM?

If yes, is there a way to change that to a shared link?

@kou
Copy link
Member

kou commented Aug 28, 2023

Does Gandiva link statically with LLVM?

Yes.

If yes, is there a way to change that to a shared link?

GH-37412

@kou kou closed this as completed in #37412 Sep 4, 2023
@kou kou added this to the 14.0.0 milestone Sep 4, 2023
kou added a commit that referenced this issue Sep 4, 2023
…37412)

### Rationale for this change

Gandiva always links LLVM statically. But we can use LLVM shared library instead to reduce `libgandiva.so` size.

### What changes are included in this PR?

Add `ARROW_LLVM_USE_SHARED` like other dependencies.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

Yes.

This has a backward incompatible change. LLVM shared library not static libraries is used by default. If you want to use LLVM static libraries, you need to specify `-DARROW_LLVM_USE_SHARED=OFF` or `-DARROW_DEPENDENCY_USE_SHARED=OFF` explicitly.
* Closes: #37410

Lead-authored-by: Sutou Kouhei <kou@clear-code.com>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
loicalleyne pushed a commit to loicalleyne/arrow that referenced this issue Nov 13, 2023
…ary (apache#37412)

### Rationale for this change

Gandiva always links LLVM statically. But we can use LLVM shared library instead to reduce `libgandiva.so` size.

### What changes are included in this PR?

Add `ARROW_LLVM_USE_SHARED` like other dependencies.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

Yes.

This has a backward incompatible change. LLVM shared library not static libraries is used by default. If you want to use LLVM static libraries, you need to specify `-DARROW_LLVM_USE_SHARED=OFF` or `-DARROW_DEPENDENCY_USE_SHARED=OFF` explicitly.
* Closes: apache#37410

Lead-authored-by: Sutou Kouhei <kou@clear-code.com>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
dgreiss pushed a commit to dgreiss/arrow that referenced this issue Feb 19, 2024
…ary (apache#37412)

### Rationale for this change

Gandiva always links LLVM statically. But we can use LLVM shared library instead to reduce `libgandiva.so` size.

### What changes are included in this PR?

Add `ARROW_LLVM_USE_SHARED` like other dependencies.

### Are these changes tested?

Yes.

### Are there any user-facing changes?

Yes.

This has a backward incompatible change. LLVM shared library not static libraries is used by default. If you want to use LLVM static libraries, you need to specify `-DARROW_LLVM_USE_SHARED=OFF` or `-DARROW_DEPENDENCY_USE_SHARED=OFF` explicitly.
* Closes: apache#37410

Lead-authored-by: Sutou Kouhei <kou@clear-code.com>
Co-authored-by: Sutou Kouhei <kou@cozmixng.org>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants