Improve blosc2 detection #1017
Merged
Improve blosc2 detection #1017
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced May 14, 2023
FrancescAlted
approved these changes
May 15, 2023
bnavigator
reviewed
May 16, 2023
Comment on lines
-101
to
+102
| # If the system provides the library and headers, | ||
| # get them from environment variables or find by pkg-config | ||
| if platform.system() in ["Linux", "Darwin"]: | ||
| try: | ||
| include_path = Path(os.environ.get( | ||
| "BLOSC2_INCDIR", | ||
| subprocess.check_output( | ||
| [PKG_CONFIG, '--variable=includedir', 'blosc2'], | ||
| text=True).strip())) | ||
| library_path = Path(os.environ.get( | ||
| "BLOSC2_LIBDIR", | ||
| subprocess.check_output( | ||
| [PKG_CONFIG, '--variable=libdir', 'blosc2'], | ||
| text=True).strip())) | ||
| except subprocess.CalledProcessError: | ||
| pass | ||
| else: | ||
| return include_path, library_path | ||
|
|
||
| # Otherwise get them from the PyPI published wheels for blosc2 | ||
| # They package the library and headers directly | ||
| try: | ||
| import blosc2 | ||
| except ModuleNotFoundError: | ||
| raise EnvironmentError("Cannot import the blosc2 requirement") | ||
| raise OSError("Cannot import the blosc2 requirement") |
There was a problem hiding this comment.
Are you sure this does not imply that the blosc2 python package is required again?
There was a problem hiding this comment.
According to the following code:
the paths coming from the blosc2 Python package (hook_dirs) are now only one of the possible candidates.
Does it work for you?
There was a problem hiding this comment.
Okay, I can confirm that the code does not halt because of above mentioned exception:
[ 5s] Processing /home/abuild/rpmbuild/BUILD/PyTables-ci-blosc2-build
[ 5s] Preparing metadata (pyproject.toml): started
[ 5s] Running command Preparing metadata (pyproject.toml)
[ 8s] * Using Python 3.9.16 (main, Dec 08 2022, 10:47:17) [GCC]
[ 8s] * Found cython 0.29.34
[ 8s] * USE_PKGCONFIG: True
[ 8s] * Found HDF5 headers at ``/usr/include``, library at ``/usr/lib64``.
[ 8s] * Found LZO 2 headers at ``/usr/include``, library at ``/usr/lib64``.
[ 8s] * Skipping detection of LZO 1 since LZO 2 has already been found.
[ 8s] * Found bzip2 headers at ``/usr/include``, library at ``/usr/lib64``.
[ 8s] * pkg-config library dirs for blosc: /usr/lib
[ 8s] * Found blosc headers at ``/usr/include``, the library is located in the standard system search dirs.
[ 8s] * Run 'blosc2_find_directories_hook'
[ 8s] * Unable to find blosc2 wheel.
[ 8s] * Found blosc2 headers at ``/usr/include``, library at ``/usr/lib64``.
However, I am seeing a failure later. Possibly because of recently released c-blosc2 2.9.1
[ 15s] gcc -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -DOPENSSL_LOAD_CONF -fwrapv -fno-semantic-interposition -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -IVendor/ -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -IVendor/ -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -flto=auto -g -fno-strict-aliasing -fPIC -DNDEBUG=1 -DHAVE_LZO2_LIB=1 -DHAVE_BZ2_LIB=1 -DHAVE_BLOSC_LIB=1 -DHAVE_BLOSC2_LIB=1 -Ihdf5-blosc2/src -Ihdf5-blosc/src -I/usr/lib64/python3.9/site-packages/numpy/core/include -I/usr/include/python3.9 -c hdf5-blosc/src/blosc_filter.c -o build/temp.linux-x86_64-cpython-39/hdf5-blosc/src/blosc_filter.o -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -flto=auto -g -fno-strict-aliasing -Isrc -DH5_USE_18_API -DH5Acreate_vers=2 -DH5Aiterate_vers=2 -DH5Dcreate_vers=2 -DH5Dopen_vers=2 -DH5Eclear_vers=2 -DH5Eprint_vers=2 -DH5Epush_vers=2 -DH5Eset_auto_vers=2 -DH5Eget_auto_vers=2 -DH5Ewalk_vers=2 -DH5E_auto_t_vers=2 -DH5Gcreate_vers=2 -DH5Gopen_vers=2 -DH5Pget_filter_vers=2 -DH5Pget_filter_by_id_vers=2 -DH5Tarray_create_vers=2 -DH5Tget_array_dims_vers=2 -DH5Z_class_t_vers=2 -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION
[ 15s] gcc -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -DOPENSSL_LOAD_CONF -fwrapv -fno-semantic-interposition -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -IVendor/ -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -IVendor/ -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -flto=auto -g -fno-strict-aliasing -fPIC -DNDEBUG=1 -DHAVE_LZO2_LIB=1 -DHAVE_BZ2_LIB=1 -DHAVE_BLOSC_LIB=1 -DHAVE_BLOSC2_LIB=1 -Ihdf5-blosc2/src -Ihdf5-blosc/src -I/usr/lib64/python3.9/site-packages/numpy/core/include -I/usr/include/python3.9 -c hdf5-blosc2/src/blosc2_filter.c -o build/temp.linux-x86_64-cpython-39/hdf5-blosc2/src/blosc2_filter.o -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -flto=auto -g -fno-strict-aliasing -Isrc -DH5_USE_18_API -DH5Acreate_vers=2 -DH5Aiterate_vers=2 -DH5Dcreate_vers=2 -DH5Dopen_vers=2 -DH5Eclear_vers=2 -DH5Eprint_vers=2 -DH5Epush_vers=2 -DH5Eset_auto_vers=2 -DH5Eget_auto_vers=2 -DH5Ewalk_vers=2 -DH5E_auto_t_vers=2 -DH5Gcreate_vers=2 -DH5Gopen_vers=2 -DH5Pget_filter_vers=2 -DH5Pget_filter_by_id_vers=2 -DH5Tarray_create_vers=2 -DH5Tget_array_dims_vers=2 -DH5Z_class_t_vers=2 -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION
[ 15s] gcc -Wno-unused-result -Wsign-compare -DNDEBUG -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -DOPENSSL_LOAD_CONF -fwrapv -fno-semantic-interposition -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -IVendor/ -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -IVendor/ -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -flto=auto -g -fno-strict-aliasing -fPIC -DNDEBUG=1 -DHAVE_LZO2_LIB=1 -DHAVE_BZ2_LIB=1 -DHAVE_BLOSC_LIB=1 -DHAVE_BLOSC2_LIB=1 -Ihdf5-blosc2/src -Ihdf5-blosc/src -I/usr/lib64/python3.9/site-packages/numpy/core/include -I/usr/include/python3.9 -c src/H5ARRAY.c -o build/temp.linux-x86_64-cpython-39/src/H5ARRAY.o -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -flto=auto -g -fno-strict-aliasing -Isrc -DH5_USE_18_API -DH5Acreate_vers=2 -DH5Aiterate_vers=2 -DH5Dcreate_vers=2 -DH5Dopen_vers=2 -DH5Eclear_vers=2 -DH5Eprint_vers=2 -DH5Epush_vers=2 -DH5Eset_auto_vers=2 -DH5Eget_auto_vers=2 -DH5Ewalk_vers=2 -DH5E_auto_t_vers=2 -DH5Gcreate_vers=2 -DH5Gopen_vers=2 -DH5Pget_filter_vers=2 -DH5Pget_filter_by_id_vers=2 -DH5Tarray_create_vers=2 -DH5Tget_array_dims_vers=2 -DH5Z_class_t_vers=2 -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION
[ 15s] In file included from hdf5-blosc2/src/blosc2_filter.h:8,
[ 15s] from src/H5ARRAY.c:8:
[ 15s] /usr/include/blosc2.h:1109:21: error: ‘BLOSC_STUNE’ undeclared here (not in a function); did you mean ‘BLOSC_LAST_TUNE’?
[ 15s] 1109 | static int g_tune = BLOSC_STUNE;
[ 15s] | ^~~~~~~~~~~
[ 15s] | BLOSC_LAST_TUNE
[ 15s] error: command '/usr/bin/gcc' failed with exit code 1
[ 15s] error: subprocess-exited-with-error
[ 15s]
[ 15s] × Building wheel for tables (pyproject.toml) did not run successfully.
[ 15s] │ exit code: 1
[ 15s] ╰─> See above for output.
[ 15s]
[ 15s] note: This error originates from a subprocess, and is likely not a problem with pip.
[ 15s] full command: /usr/bin/python3.9 /usr/lib/python3.9/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmp1tebzobf
[ 15s] cwd: /home/abuild/rpmbuild/BUILD/PyTables-ci-blosc2-build
[ 15s] Building wheel for tables (pyproject.toml): finished with status 'error'
[ 15s] ERROR: Failed building wheel for tables
[ 15s] Failed to build tables
I'll investigate further and may open another issue if necessary
There was a problem hiding this comment.
Nevermind, I saw you blacklisted the python-blosc2 equivalent in 482ef44
There was a problem hiding this comment.
yes, indeed
I forgot to mention
avalentino
force-pushed
the
ci/blosc2-build
branch
from
8295cfd
to
1fb8168
Compare
avalentino
force-pushed
the
ci/blosc2-build
branch
from
1fb8168
to
cec41fe
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #998 and #1016.