Symbol not found: _SZ_BufftoBuffCompress

[dipterix]

https://cran.r-project.org/web/checks/check_results_hdf5r.html

Cannot install on macOS or Solaris.

I also encountered issue when checking with rhub

rhub::check(platform = "macos-highsierra-release-cran")

Check results for the package raveio which imports hdf5r: (the link might not be permanent)

• Fail to install on macOS

Error summaries:

Error: package or namespace load failed for ‘hdf5r’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/Users/userdggyP9YY/R/00LOCK-hdf5r/00new/hdf5r/libs/hdf5r.so':
  dlopen(/Users/userdggyP9YY/R/00LOCK-hdf5r/00new/hdf5r/libs/hdf5r.so, 6): Symbol not found: _SZ_BufftoBuffCompress
  Referenced from: /Users/userdggyP9YY/R/00LOCK-hdf5r/00new/hdf5r/libs/hdf5r.so
  Expected in: flat namespace
 in /Users/userdggyP9YY/R/00LOCK-hdf5r/00new/hdf5r/libs/hdf5r.so
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/Users/userdggyP9YY/R/hdf5r’

[hhoeflin]

Thank you for letting me know. Unfortunately I don't have access to either Solaris nor a latest version Mac, so cannot really debug this issue. If you have a PR that resolves this issue, that would be much appreciated and I would merge it.

[beauchamplab]

Unfortunately I don't know too much about coding in C.

However, I guess you could try rhub to check the problem without bothering CRAN. I was able to reproduce the "flat namespace" on macOS (Symbol not found: _SZ_BufftoBuffCompress) by

rhub::check(platform = "macos-highsierra-release-cran")

I guess they have very unique setups for macOS and Solaris.

Additional link:
https://www.r-project.org/nosvn/R.check/r-release-macos-x86_64/hdf5r-00install.html

[hhoeflin]

Thanks for your suggestion. I think this may be related to a missing library on the newer MacOS. Someone with a Mac that can install dependencies needs to check this.

[ShotaOchi]

I think you don't have to do something to fix the error on macOS if CRAN doesn't tell you to fix it.

The CRAN environment of macOS is really fragile.
For example, the CRAN Package Check Results for Package magickGUI often shows nasty warnings.
https://cran.r-project.org/web/checks/check_results_magickGUI.html
Sometimes the warnings were fixed but the warnings appear again and again.
I sent an email about the warnings to CRAN but CRAN did nothing.
CRAN says nothing about the warnings and magickGUI package is retained in CRAN safely.

It's a waste of time to struggle with the weird CRAN environment.

[grimbough]

Arriving here from https://forum.hdfgroup.org/t/symbol-not-found-sz-bufftobuffcompress/8154

AFAIK that error is produced when you don't link again the SZIP library, and I don't see the -lsz flag in that 00install example. It looks like hdf5r does a nice job of getting the linking flags from the local h5cc, so it seems weird that it wouldn't have that flag if some part had been built with SZIP support turned on. Maybe you know if there's some part of hdf5r that assumes SZIP will be present?

---

As a counterpoint to @ShotaOchi I regularly get emails from the CRAN maintainers if rhdf5 or Rhdf5lib breaks on Mac or Solaris, and they're not even a CRAN packages! Sorry that's not very helpful, but I can fire up my Mac virtual machine and see if there's anything obvious about the default HDF5 installation that might be triggering this.

[dipterix]

Yeah sometimes CRAN issues do automatically fix by themselves. But it's tricky because those issues mean there are some portability problems either in your package or the dependency. And your package cannot properly find the non basic system utilities properly.

CRAN has some people manually debugging those problems. You receive those emails when they can find out the bugs.

I opened this issue because my package that depends on hdf5r couldn't get through all the tests.

[ShotaOchi]

My comment has a bad vibe but blaming CRAN is not my purpose.
What I wanted to say is it's not time to try to fix the error if CRAN says nothing.

[jeroen]

The CRAN error on macos Symbol not found: _SZ_BufftoBuffCompress probably means your package is not linking to szip. You need to add -lz to PKG_LIBS on MacOS.

[jeroen]

The problem seems to be that h5cc gives us the correct flags:

# h5cc -show
clang -I/usr/local/opt/szip/include -L/usr/local/Cellar/hdf5/1.12.1/lib /usr/local/Cellar/hdf5/1.12.1/lib/libhdf5_hl.a /usr/local/Cellar/hdf5/1.12.1/lib/libhdf5.a -L/usr/local/opt/szip/lib -lsz -lz -ldl -lm

However your autoconf script seems to ignore all that and always link the R package with this hardcoded line:

hdf5r/configure.ac

Line 112 in 7e2d84d

HDF5R_LIBS="${LIBS} -L. -lhdf5_hl -lhdf5 -lz -lm"

And because that line is missing -lsz you are not linking to szip.

[hhoeflin]

I unfortunately don't have a Mac, but if you could make a PR I am happy to merge it. Thanks

[jeroen]

OK I have sent a PR: #180
PS you can easily test this by installing the r-universe app on your github account: https://github.com/apps/r-universe/installations/new

[hhoeflin]

The new version 1.3.4 is on CRAN with fix #180, but the error on new MacOS ARM still occurs:

https://www.r-project.org/nosvn/R.check/r-release-macos-arm64/hdf5r-00check.html

[jeroen]

CRAN has not yet updated the arm check for your package, sometimes this takes a few days. The log file you linked above still mentions hdf5r 1.3.3 at this point. It may take longer because the maintainers are currently on vacation.

[hhoeflin]

Oh sorry.Thanks i didn't notice.

…

On Mon, Aug 30, 2021, 10:50 Jeroen Ooms ***@***.***> wrote: CRAN has not yet updated the arm check for your package, sometimes this takes a few days. The log file you linked above still mentions hdf5r 1.3.3 at this point. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#165 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AASGMYWUMMYYZO2JOKU7SC3T7NA4NANCNFSM4XTAY3DQ> . Triage notifications on the go with GitHub Mobile for iOS <https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[hhoeflin]

Checks now passing except for Solaris. Closing this issue.

[dipterix]

Sorry to post it again here after 3 years. I was trying to install hdf5r on r-devel when this happened:

checking for HDF5 libraries... clang: error: unknown argument: '-show'
clang: error: unknown argument: '-show'
yes (version 1.14.3)

It looks like running h5cc -show has been changed, hence the configure.ac does not work anymore...

I'm compiling with HomeBrew installed HDF5 library.

[hhoeflin]

Could you send me what h5cc -show shows for you under 1.14.3 and what it shows for a version that is still working before?

[dipterix]

There is no such option...

% h5cc -show
/opt/homebrew/h5cc
dir is /opt/homebrew
clang: error: unknown argument: '-show'

I think this is because the Homebrew builds HDF5 in autotools rather than cmake.

However, they do have pkg-config .pc file (at some point around 2010, they made sure the pkg-config always exists).

Here are flags using pkg-config

% pkg-config --libs hdf5 
-L/opt/homebrew/Cellar/hdf5/1.14.3_1/lib -lhdf5
% pkg-config --cflags hdf5
-I/opt/homebrew/Cellar/hdf5/1.14.3_1/include

[hhoeflin]

This HDFGroup/hdf5#8 seems to indicate the the lack of pkg-config is a longstanding issue and is a difference between cmake and autotools builds. In my autotools-based hdf5, there is no pkgconfig-file included and as the issue states, some of this also differs by distro.

In any case, maybe this should be a separate new issue, not this old one.

See also discussion here: spack/spack#27000

[hhoeflin]

Also Homebrew/homebrew-core#159691