ERROR:: Could not find a local HDF5 installation. #4162

Michel-Settembrino · 2021-01-07T16:12:01Z

Describe your environment

Operating system: macOS Big Sur Version 11.0.1
Python Version: 2.7.16 (python -V)
Python3 Version: 3.9.1 (python3 -V)
CCXT version: _____ (pip freeze | grep ccxt)
Freqtrade Version: ____ (freqtrade -V or docker-compose run --rm freqtrade -V for Freqtrade running in docker)

Note: All issues other than enhancement requests will be closed without further comment if the above template is deleted or not filled out.

Describe the problem:

Explain the problem you have encountered

Error during installation using default script ./setup.sh --install

Steps to reproduce:

./setup.sh --install

Observed Results:

What happened?
Installation failed.
What did you expect to happen?
A running installation.

Relevant code exceptions or logs

Note: Please copy/paste text of the messages, no screenshots of logs please.

Collecting tables==3.6.1
Using cached tables-3.6.1.tar.gz (4.6 MB)
  ERROR: Command errored out with exit status 1:
   command: /Users/michel/Crypto-bots/freqtrade/.env/bin/python3.9 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/sv/f00rtph94cj0k1ntrvxxpsnh0000gn/T/pip-install-m6ukwy7f/tables_ac85413363944fa59515c6933d5c3876/setup.py'"'"'; __file__='"'"'/private/var/folders/sv/f00rtph94cj0k1ntrvxxpsnh0000gn/T/pip-install-m6ukwy7f/tables_ac85413363944fa59515c6933d5c3876/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /private/var/folders/sv/f00rtph94cj0k1ntrvxxpsnh0000gn/T/pip-pip-egg-info-aqdglu39
       cwd: /private/var/folders/sv/f00rtph94cj0k1ntrvxxpsnh0000gn/T/pip-install-m6ukwy7f/tables_ac85413363944fa59515c6933d5c3876/
  Complete output (7 lines):
  clang: error: invalid version number in 'MACOSX_DEPLOYMENT_TARGET=11'
  * Using Python 3.9.1 (default, Dec 29 2020, 09:45:39)
  * USE_PKGCONFIG: True
  .. ERROR:: Could not find a local HDF5 installation.
     You may need to explicitly state where your local HDF5 headers and
     library can be found by setting the ``HDF5_DIR`` environment
     variable or by using the ``--hdf5`` command-line option.
  ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

The text was updated successfully, but these errors were encountered:

xmatthias · 2021-01-07T16:21:08Z

To be honest, it's a known issue with macOS and python 3.9.

I've opened an issue a while ago requesting wheels for it so the base dependency is not needed - but so far they've not been created (not for MacOS, anyway).

Feel free to submit to the issue i opened (PyTables/PyTables#823) for updates on it.

I guess you can solve this somehow by installing a hdf5 binary to your system so it's available for the compilation process.
However, since i don't own a mac, i will not be able to support (maybe check in the PyTables repository, they may have some answers / guides how to install from sources on a mac).

Alternatively, use python 3.8 for now, which does have wheels available, and should therefore install without going through this hastle.

edit
based on their readme, it should be brew install hdf5 - followed by trying the installation again.

Let me know if that works for you.

edit2 maybe also look at PyTables/PyTables#828 - if the steps described there solve this.

xmatthias · 2021-01-07T18:45:39Z

Please try to run brew install hdf5 c-blosc before starting the installation.

It seems to fix the problem in CI ... however i THINK that's a slightly older version of macOS...

Michel-Settembrino · 2021-01-08T13:53:36Z

I followed your first suggestion by first installing hdf5 using command brew install hdf5 and restart the installation.
The installation of hdf5 went good but not the installation of freqtrade. I have another issue.
This is the log:

Collecting webencodings
Downloading webencodings-0.5.1-py2.py3-none-any.whl (11 kB)
Using legacy 'setup.py install' for sdnotify, since package 'wheel' is not installed.
Using legacy 'setup.py install' for TA-Lib, since package 'wheel' is not installed.
Using legacy 'setup.py install' for tables, since package 'wheel' is not installed.
Using legacy 'setup.py install' for wrapt, since package 'wheel' is not installed.
Using legacy 'setup.py install' for yarl, since package 'wheel' is not installed.
Using legacy 'setup.py install' for docopt, since package 'wheel' is not installed.
Using legacy 'setup.py install' for MarkupSafe, since package 'wheel' is not installed.
Using legacy 'setup.py install' for pandocfilters, since package 'wheel' is not installed.
Using legacy 'setup.py install' for pycares, since package 'wheel' is not installed.
Using legacy 'setup.py install' for pyrsistent, since package 'wheel' is not installed.
Using legacy 'setup.py install' for retrying, since package 'wheel' is not installed.
Using legacy 'setup.py install' for PyYAML, since package 'wheel' is not installed.
Building wheels for collected packages: py-find-1st
Building wheel for py-find-1st (PEP 517) ... error
ERROR: Command errored out with exit status 1:
 command: /Users/michel/Crypto-bots/freqtrade/.env/bin/python3.9 /Users/michel/Crypto-bots/freqtrade/.env/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /var/folders/sv/f00rtph94cj0k1ntrvxxpsnh0000gn/T/tmp55cat798
     cwd: /private/var/folders/sv/f00rtph94cj0k1ntrvxxpsnh0000gn/T/pip-install-ig_77xfn/py-find-1st_37fb44144e2c4060844ac1a947852e8c
Complete output (15 lines):
running bdist_wheel
running build
running build_py
creating build
creating build/lib.macosx-11-x86_64-3.9
creating build/lib.macosx-11-x86_64-3.9/utils_find_1st
copying utils_find_1st/__init__.py -> build/lib.macosx-11-x86_64-3.9/utils_find_1st
running build_ext
check for clang compiler ... yes
building 'find_1st' extension
creating build/temp.macosx-11-x86_64-3.9
creating build/temp.macosx-11-x86_64-3.9/utils_find_1st
clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -I/usr/local/include -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -DNPY_NO_DEPRECATED_API=NPY_1_13_API_VERSION -I/private/var/folders/sv/f00rtph94cj0k1ntrvxxpsnh0000gn/T/pip-build-env-3cp9364_/overlay/lib/python3.9/site-packages/numpy/core/include -I/usr/local/include -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/sqlite/include -I/Users/michel/Crypto-bots/freqtrade/.env/include -I/usr/local/Cellar/python@3.9/3.9.1_4/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c utils_find_1st/find_1st.cpp -o build/temp.macosx-11-x86_64-3.9/utils_find_1st/find_1st.o -Wno-unused-function
clang: error: invalid version number in 'MACOSX_DEPLOYMENT_TARGET=11'
error: command '/usr/bin/clang' failed with exit code 1
----------------------------------------
ERROR: Failed building wheel for py-find-1st
Failed to build py-find-1st
ERROR: Could not build wheels for py-find-1st which use PEP 517 and cannot be installed directly
Failed installing dependencies

Michel-Settembrino · 2021-01-08T13:59:41Z

I also tried instructions from PyTables/PyTables#828 but second command fails.

brew install c-blosc
pip3 install tables

Michel-Settembrino · 2021-01-08T14:34:01Z

I also tried brew install hdf5 c-blosc but another issue during install of freqtrade

Using legacy 'setup.py install' for PyYAML, since package 'wheel' is not installed.
Building wheels for collected packages: py-find-1st
  Building wheel for py-find-1st (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /Users/michel/Crypto-bots/freqtrade/.env/bin/python3.9 /Users/michel/Crypto-bots/freqtrade/.env/lib/python3.9/site-packages/pip/_vendor/pep517/_in_process.py build_wheel /var/folders/sv/f00rtph94cj0k1ntrvxxpsnh0000gn/T/tmp4a_gw10y
       cwd: /private/var/folders/sv/f00rtph94cj0k1ntrvxxpsnh0000gn/T/pip-install-t257egie/py-find-1st_2d901bf5f67847ba9f331c2347dc1d95
  Complete output (15 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.macosx-11-x86_64-3.9
  creating build/lib.macosx-11-x86_64-3.9/utils_find_1st
  copying utils_find_1st/__init__.py -> build/lib.macosx-11-x86_64-3.9/utils_find_1st
  running build_ext
  check for clang compiler ... yes
  building 'find_1st' extension
  creating build/temp.macosx-11-x86_64-3.9
  creating build/temp.macosx-11-x86_64-3.9/utils_find_1st
  clang -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -I/usr/local/include -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -DNPY_NO_DEPRECATED_API=NPY_1_13_API_VERSION -I/private/var/folders/sv/f00rtph94cj0k1ntrvxxpsnh0000gn/T/pip-build-env-qawbn8mo/overlay/lib/python3.9/site-packages/numpy/core/include -I/usr/local/include -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/sqlite/include -I/Users/michel/Crypto-bots/freqtrade/.env/include -I/usr/local/Cellar/python@3.9/3.9.1_4/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c utils_find_1st/find_1st.cpp -o build/temp.macosx-11-x86_64-3.9/utils_find_1st/find_1st.o -Wno-unused-function
  clang: error: invalid version number in 'MACOSX_DEPLOYMENT_TARGET=11'
  error: command '/usr/bin/clang' failed with exit code 1
  ----------------------------------------
  ERROR: Failed building wheel for py-find-1st
Failed to build py-find-1st
ERROR: Could not build wheels for py-find-1st which use PEP 517 and cannot be installed directly
Failed installing dependencies

Michel-Settembrino · 2021-01-08T18:08:25Z

I got my issue solved and could install freqtrade.

Solution:

brew install hdf5
Update the XCode Command Line Tools to version 12.3
This is to solve an issue with clang clang: error: invalid version number in 'MACOSX_DEPLOYMENT_TARGET=11'
see this post: https://emilwypych.com/2020/11/23/pyenv-problem-macosx_deployment_target11-0/?cn-reloaded=1 for more info.
./setup.sh --install

camxus · 2021-07-31T17:16:15Z

Solution

pip install cython

brew install hdf5

brew install c-blosc

export HDF5_DIR=/usr/local/

export BLOSC_DIR=/usr/local/

pip install tables

pellet · 2021-10-30T07:13:47Z

I tried installing tables under ARM64(M1) but didn't have any luck, needed to follow @camxus directions under rosetta then it worked :)

xmatthias · 2021-10-30T07:33:51Z

The recommended way to run freqtrade is using docker, where we also provide images for ARM64.

Native installation will be troublesome due to some dependencies, and support can best be found in each dependencies dedicated repository.

Michel-Settembrino added the Triage Needed Issues yet to verify label Jan 7, 2021

xmatthias added the Install Problems with installation label Jan 7, 2021

xmatthias removed the Triage Needed Issues yet to verify label Jan 7, 2021

xmatthias mentioned this issue Jan 7, 2021

Run CI for mac on 3.9 #4164

Merged

xmatthias closed this as completed in #4164 Jan 8, 2021

samgermain mentioned this issue Jul 15, 2021

setup.sh mac fix #5271

Merged

Kaw11 mentioned this issue Sep 19, 2021

ERROR:: Could not find a local HDF5 installation - WINDOWS issue #5597

Closed

fierytortilla mentioned this issue Oct 7, 2021

Difficulty in getting any results in any experiment, even with the provided sample run MannLabs/alphapept#354

Closed

freqtrade locked as resolved and limited conversation to collaborators Oct 30, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

ERROR:: Could not find a local HDF5 installation. #4162

ERROR:: Could not find a local HDF5 installation. #4162

Michel-Settembrino commented Jan 7, 2021

xmatthias commented Jan 7, 2021 •

edited

xmatthias commented Jan 7, 2021

Michel-Settembrino commented Jan 8, 2021 •

edited

Michel-Settembrino commented Jan 8, 2021 •

edited

Michel-Settembrino commented Jan 8, 2021 •

edited

Michel-Settembrino commented Jan 8, 2021

camxus commented Jul 31, 2021 •

edited

pellet commented Oct 30, 2021

xmatthias commented Oct 30, 2021

ERROR:: Could not find a local HDF5 installation. #4162

ERROR:: Could not find a local HDF5 installation. #4162

Comments

Michel-Settembrino commented Jan 7, 2021

Describe your environment

Describe the problem:

Steps to reproduce:

Observed Results:

Relevant code exceptions or logs

xmatthias commented Jan 7, 2021 • edited

xmatthias commented Jan 7, 2021

Michel-Settembrino commented Jan 8, 2021 • edited

Michel-Settembrino commented Jan 8, 2021 • edited

Michel-Settembrino commented Jan 8, 2021 • edited

Michel-Settembrino commented Jan 8, 2021

camxus commented Jul 31, 2021 • edited

pellet commented Oct 30, 2021

xmatthias commented Oct 30, 2021

xmatthias commented Jan 7, 2021 •

edited

Michel-Settembrino commented Jan 8, 2021 •

edited

Michel-Settembrino commented Jan 8, 2021 •

edited

Michel-Settembrino commented Jan 8, 2021 •

edited

camxus commented Jul 31, 2021 •

edited