Numpy issue on Rasberry pi 4 (model B)

[Therval]

• Operating system: Raspbian buster on Pi 4
• Branch: Master

I tried to install freqtrade my raspberry pi 4 only, I came across this problem during the installation and I do not know how to solve it ..
I have already deleted the entire installation, tried again and I still have a problem with the numpy package without knowing why or how to fix it.

Despite this error, the program starts the configuration of the bot by asking me my parameters but I guess it will not work properly with this error?

Thank you.

PS : I use the easy installation linux script + raspbian note

Relevant code exceptions or logs:

ERROR: Command errored out with exit status 1:
     command: /home/pi/Documents/freqtrade/freqtrade-master/.env/bin/python3.7 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-742vklm2/py-find-1st/setup.py'"'"'; __file__='"'"'/tmp/pip-install-742vklm2/py-find-1st/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base pip-egg-info
         cwd: /tmp/pip-install-742vklm2/py-find-1st/
    Complete output (45 lines):
    Traceback (most recent call last):
      File "/home/pi/Documents/freqtrade/freqtrade-master/.env/lib/python3.7/site-packages/numpy/core/__init__.py", line 17, in <module>
        from . import multiarray
      File "/home/pi/Documents/freqtrade/freqtrade-master/.env/lib/python3.7/site-packages/numpy/core/multiarray.py", line 14, in <module>
        from . import overrides
      File "/home/pi/Documents/freqtrade/freqtrade-master/.env/lib/python3.7/site-packages/numpy/core/overrides.py", line 7, in <module>
        from numpy.core._multiarray_umath import (
    ImportError: libf77blas.so.3: cannot open shared object file: No such file or directory
    
    During handling of the above exception, another exception occurred:
    
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
      File "/tmp/pip-install-742vklm2/py-find-1st/setup.py", line 10, in <module>
        import numpy as np
      File "/home/pi/Documents/freqtrade/freqtrade-master/.env/lib/python3.7/site-packages/numpy/__init__.py", line 142, in <module>
        from . import core
      File "/home/pi/Documents/freqtrade/freqtrade-master/.env/lib/python3.7/site-packages/numpy/core/__init__.py", line 47, in <module>
        raise ImportError(msg)
    ImportError:
    
    IMPORTANT: PLEASE READ THIS FOR ADVICE ON HOW TO SOLVE THIS ISSUE!
    
    Importing the numpy c-extensions failed.
    - Try uninstalling and reinstalling numpy.
    - If you have already done that, then:
      1. Check that you expected to use Python3.7 from "/home/pi/Documents/freqtrade/freqtrade-master/.env/bin/python3.7",
         and that you have no directories in your PATH or PYTHONPATH that can
         interfere with the Python and numpy version "1.17.0" you're trying to use.
      2. If (1) looks fine, you can open a new issue at
         https://github.com/numpy/numpy/issues.  Please include details on:
         - how you installed Python
         - how you installed numpy
         - your operating system
         - whether or not you have multiple versions of Python installed
         - if you built from source, your compiler versions and ideally a build log
    
    - If you're working with a numpy git repository, try `git clean -xdf`
      (removes all files not under version control) and rebuild numpy.
    
    Note: this error has many possible causes, so please don't comment on
    an existing issue about this - open a new one instead.
    
    Original error was: libf77blas.so.3: cannot open shared object file: No such file or directory
    
    ----------------------------------------
ERROR: Command errored out with exit status 1: python setup.py egg_info Check the logs for full command output.

[xmatthias]

I don't think setup.sh and the raspbian install method work well together (one uses a python virtual environment, while the other uses conda to install compiled dependencies).

The error you're getting seems to be comming from a missing dependency according to this numpy issue (from a pi zero, but it should be the same for all raspbian devices) numpy/numpy#11110

Try their fix...sudo apt-get install libatlas-base-dev please and let us know

[Therval]

thank a lot!
I'll try tonight!

Which installation method would be best? The one on freqtrade.io starting with the note on Raspbian and continuing with the Common part?

[xmatthias]

for a Raspberry PI, i would install ta-lib (the c library) regularily (you have to), and then do the raspbian instructions from the documentation.

Otherwise, the installation takes half a day or so (at least on a RPI3 - i don't own a RPI4 yet).

[Therval]

So now, i have 2 issues..
1 with scipy :

ERROR: Failed building wheel for scipy
  Running setup.py clean for scipy
  ERROR: Command errored out with exit status 1:
   command: /home/pi/miniconda3/envs/freqtrade/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-s2bwtgt4/scipy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-s2bwtgt4/scipy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' clean --all
       cwd: /tmp/pip-install-s2bwtgt4/scipy
  Complete output (9 lines):
  
  `setup.py clean` is not supported, use one of the following instead:
  
    - `git clean -xdf` (cleans all files)
    - `git clean -Xdf` (cleans all versioned files, doesn't touch
                        files that aren't checked into the git repo)
  
  Add `--force` to your command to use it anyway if you must (unsupported).
  
  ----------------------------------------
  ERROR: Failed cleaning build dir for scipy

and another with scikit-learn :

 Building wheel for scikit-learn (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/pi/miniconda3/envs/freqtrade/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-s2bwtgt4/scikit-learn/setup.py'"'"'; __file__='"'"'/tmp/pip-install-s2bwtgt4/scikit-learn/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-psvj285a --python-tag cp36
       cwd: /tmp/pip-install-s2bwtgt4/scikit-learn/
  Complete output (28 lines):
  Partial import of sklearn during the build process.
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
    File "/tmp/pip-install-s2bwtgt4/scikit-learn/setup.py", line 290, in <module>
      setup_package()
    File "/tmp/pip-install-s2bwtgt4/scikit-learn/setup.py", line 286, in setup_package
      setup(**metadata)
    File "/home/pi/miniconda3/envs/freqtrade/lib/python3.6/site-packages/numpy/distutils/core.py", line 135, in setup
      config = configuration()
    File "/tmp/pip-install-s2bwtgt4/scikit-learn/setup.py", line 174, in configuration
      config.add_subpackage('sklearn')
    File "/home/pi/miniconda3/envs/freqtrade/lib/python3.6/site-packages/numpy/distutils/misc_util.py", line 1024, in add_subpackage
      caller_level = 2)
    File "/home/pi/miniconda3/envs/freqtrade/lib/python3.6/site-packages/numpy/distutils/misc_util.py", line 993, in get_subpackage
      caller_level = caller_level + 1)
    File "/home/pi/miniconda3/envs/freqtrade/lib/python3.6/site-packages/numpy/distutils/misc_util.py", line 930, in _get_configuration_from_setup_py
      config = setup_module.configuration(*args)
    File "sklearn/setup.py", line 62, in configuration
      config.add_subpackage('utils')
    File "/home/pi/miniconda3/envs/freqtrade/lib/python3.6/site-packages/numpy/distutils/misc_util.py", line 1024, in add_subpackage
      caller_level = 2)
    File "/home/pi/miniconda3/envs/freqtrade/lib/python3.6/site-packages/numpy/distutils/misc_util.py", line 993, in get_subpackage
      caller_level = caller_level + 1)
    File "/home/pi/miniconda3/envs/freqtrade/lib/python3.6/site-packages/numpy/distutils/misc_util.py", line 930, in _get_configuration_from_setup_py
      config = setup_module.configuration(*args)
    File "sklearn/utils/setup.py", line 8, in configuration
      from Cython import Tempita
  ModuleNotFoundError: No module named 'Cython'
  ----------------------------------------
  ERROR: Failed building wheel for scikit-learn

[Therval]

I do not know why I have so much trouble, when I rule, two others appear

[Therval]

At the end of the line python3 -m pip install -r requirements.txt, when executed, i have

Failed to build scikit-learn
ERROR: Could not build wheels for scipy which use PEP 517 and cannot be installed directly

[Therval]

I execute the line python3 -m pip install -e . to see what happen but i expect some issues too..

[xmatthias]

I think you should follow these steps...

copy/pasted for clarity:

sudo apt install libffi-dev

conda config --add channels rpi
conda install python=3.6
conda create -n freqtrade python=3.6
conda activate freqtrade
conda install scipy pandas numpy

sudo apt install libffi-dev
python3 -m pip install -r requirements-common.txt
python3 -m pip install -e .

Run the line by line ... the only prerequisite is to have ta-lib and (mini/berry?)conda installed
scipy should not build a wheel but download a compiled version from conda ... compiling on raspberries is not always fun ...

[Therval]

Good! I finally managed to install it.
I followed your procedure but I still faced the same error for scikit-learn. To solve it I had to install the manual Cython package and then the installation went smoothly.

I put the code here for those who have an RPi and who have the same problem.

sudo apt install libffi-dev

conda config --add channels rpi
conda install python=3.6
conda create -n freqtrade python=3.6
conda activate freqtrade
conda install scipy pandas numpy
sudo apt-get install libatlas-base-dev
pip install cython

python3 -m pip install -r requirements-common.txt
python3 -m pip install -e .

[Therval]

Just one last question, it is possible to put freqtrade as a service on the raspberry?

[xmatthias]

yes this should be possible.

https://www.freqtrade.io/en/latest/installation/#7-optional-configure-freqtrade-as-a-systemd-service should work for this case too - however you'll need to modify the ExecStart= command to match the path of freqtrade (run which freqtrade to get the path within the virtual environment.

[Therval]

Thanks a lot for your help !