Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug]: Can't install scipy and PyFFTW because of numpy problems. #14095

Closed
tigran123 opened this issue Dec 24, 2022 · 10 comments
Closed

[Bug]: Can't install scipy and PyFFTW because of numpy problems. #14095

tigran123 opened this issue Dec 24, 2022 · 10 comments
Labels
bug report Something is not working properly. python Issue is about python-related stuff, including pip.

Comments

@tigran123
Copy link

Problem description

Trying to install scipy or PyFFTW python packages fails.

What steps will reproduce the bug?

  1. Run pip install scipy
  2. Observe the following error:
$ pip install scipy
Collecting scipy
  Using cached scipy-1.9.3.tar.gz (42.1 MB)
  Installing build dependencies ... error
  error: subprocess-exited-with-error
  
  × pip subprocess to install build dependencies did not run successfully.
  │ exit code: 1
  ╰─> [266 lines of output]
      Ignoring numpy: markers 'python_version == "3.8" and platform_machine == "aarch64" and platform_python_implementation != "PyPy"' don't match your environment
      Ignoring numpy: markers 'python_version == "3.8" and platform_machine == "arm64" and platform_system == "Darwin"' don't match your environment
      Ignoring numpy: markers 'python_version == "3.9" and platform_machine == "arm64" and platform_system == "Darwin"' don't match your environment
      Ignoring numpy: markers 'platform_machine == "loongarch64"' don't match your environment
      Ignoring numpy: markers 'python_version == "3.10" and platform_system == "Windows" and platform_python_implementation != "PyPy"' don't match your environment
      Ignoring numpy: markers 'python_version == "3.8" and (platform_machine != "arm64" or platform_system != "Darwin") and platform_machine != "aarch64" and platform_machine != "loongarch64" and platform_python_implementation != "PyPy"' don't match your environment
      Ignoring numpy: markers 'python_version == "3.9" and (platform_machine != "arm64" or platform_system != "Darwin") and platform_machine != "loongarch64" and platform_python_implementation != "PyPy"' don't match your environment
      Ignoring numpy: markers 'python_version == "3.10" and (platform_system != "Windows" and platform_machine != "loongarch64") and platform_python_implementation != "PyPy"' don't match your environment
      Ignoring numpy: markers 'python_version >= "3.12"' don't match your environment
      Ignoring numpy: markers 'python_version >= "3.8" and platform_python_implementation == "PyPy"' don't match your environment
      Collecting meson-python>=0.9.0
        Using cached meson_python-0.12.0-py3-none-any.whl (66 kB)
      Collecting Cython<3.0,>=0.29.32
        Using cached Cython-0.29.32-py2.py3-none-any.whl (986 kB)
      Collecting pybind11<2.11.0,>=2.4.3
        Using cached pybind11-2.10.2-py3-none-any.whl (222 kB)
      Collecting pythran<0.13.0,>=0.9.12
        Using cached pythran-0.12.0-py3-none-any.whl (4.2 MB)
      Collecting wheel<0.38.0
        Using cached wheel-0.37.1-py2.py3-none-any.whl (35 kB)
      Collecting numpy==1.23.2
        Using cached numpy-1.23.2.tar.gz (10.7 MB)
        Installing build dependencies: started
        Installing build dependencies: finished with status 'done'
        Getting requirements to build wheel: started
        Getting requirements to build wheel: finished with status 'done'
        Preparing metadata (pyproject.toml): started
        Preparing metadata (pyproject.toml): finished with status 'done'
      Collecting meson>=0.63.3
        Using cached meson-1.0.0-py3-none-any.whl (900 kB)
      Collecting pyproject-metadata>=0.6.1
        Using cached pyproject_metadata-0.6.1-py3-none-any.whl (7.4 kB)
      Collecting ply>=3.4
        Using cached ply-3.11-py2.py3-none-any.whl (49 kB)
      Collecting gast~=0.5.0
        Using cached gast-0.5.3-py3-none-any.whl (19 kB)
      Collecting beniget~=0.4.0
        Using cached beniget-0.4.1-py3-none-any.whl (9.4 kB)
      Collecting packaging>=19.0
        Using cached packaging-22.0-py3-none-any.whl (42 kB)
      Building wheels for collected packages: numpy
        Building wheel for numpy (pyproject.toml): started
        Building wheel for numpy (pyproject.toml): finished with status 'error'
        error: subprocess-exited-with-error
      
        × Building wheel for numpy (pyproject.toml) did not run successfully.
        │ exit code: 1
        ╰─> [212 lines of output]
            setup.py:71: RuntimeWarning: NumPy 1.23.2 may not yet support Python 3.11.
              warnings.warn(
            Running from numpy source directory.
            setup.py:86: DeprecationWarning:
      
              `numpy.distutils` is deprecated since NumPy 1.23.0, as a result
              of the deprecation of `distutils` itself. It will be removed for
              Python >= 3.12. For older Python versions it will remain present.
              It is recommended to use `setuptools < 60.0` for those Python versions.
              For more details, see:
                https://numpy.org/devdocs/reference/distutils_status_migration.html
      
      
              import numpy.distutils.command.sdist
            Processing numpy/random/_bounded_integers.pxd.in
            Processing numpy/random/_pcg64.pyx
            Processing numpy/random/_common.pyx
            Processing numpy/random/_generator.pyx
            Processing numpy/random/mtrand.pyx
            Processing numpy/random/_bounded_integers.pyx.in
            Processing numpy/random/_philox.pyx
            Processing numpy/random/bit_generator.pyx
            Processing numpy/random/_mt19937.pyx
            Processing numpy/random/_sfc64.pyx
            Cythonizing sources
            INFO: blas_opt_info:
            INFO: blas_armpl_info:
            INFO: customize UnixCCompiler
            INFO:   libraries armpl_lp64_mp not found in ['/data/data/com.termux/files/usr/lib']
            INFO:   NOT AVAILABLE
            INFO:
            INFO: blas_mkl_info:
            INFO:   libraries mkl_rt not found in ['/data/data/com.termux/files/usr/lib']
            INFO:   NOT AVAILABLE
            INFO:
            INFO: blis_info:
            INFO:   libraries blis not found in ['/data/data/com.termux/files/usr/lib']
            INFO:   NOT AVAILABLE
            INFO:
            INFO: openblas_info:
            INFO: C compiler: aarch64-linux-android-clang -DNDEBUG -g -fwrapv -O3 -Wall -fstack-protector-strong -O3 -fstack-protector-strong -O3 -fPIC
      
            creating /data/data/com.termux/files/usr/tmp/tmpcmsmdn84/data
            creating /data/data/com.termux/files/usr/tmp/tmpcmsmdn84/data/data
            creating /data/data/com.termux/files/usr/tmp/tmpcmsmdn84/data/data/com.termux
            creating /data/data/com.termux/files/usr/tmp/tmpcmsmdn84/data/data/com.termux/files
            creating /data/data/com.termux/files/usr/tmp/tmpcmsmdn84/data/data/com.termux/files/usr
            creating /data/data/com.termux/files/usr/tmp/tmpcmsmdn84/data/data/com.termux/files/usr/tmp
            creating /data/data/com.termux/files/usr/tmp/tmpcmsmdn84/data/data/com.termux/files/usr/tmp/tmpcmsmdn84
            INFO: compile options: '-c'
            INFO: aarch64-linux-android-clang: /data/data/com.termux/files/usr/tmp/tmpcmsmdn84/source.c
            INFO: aarch64-linux-android-clang /data/data/com.termux/files/usr/tmp/tmpcmsmdn84/data/data/com.termux/files/usr/tmp/tmpcmsmdn84/source.o -L/data/data/com.termux/files/usr/lib -lopenblas -o /data/data/com.termux/files/usr/tmp/tmpcmsmdn84/a.out
            INFO:   FOUND:
            INFO:     libraries = ['openblas', 'openblas']
            INFO:     library_dirs = ['/data/data/com.termux/files/usr/lib']
            INFO:     language = c
            INFO:     define_macros = [('HAVE_CBLAS', None)]
            INFO:
            INFO:   FOUND:
            INFO:     libraries = ['openblas', 'openblas']
            INFO:     library_dirs = ['/data/data/com.termux/files/usr/lib']
            INFO:     language = c
            INFO:     define_macros = [('HAVE_CBLAS', None)]
            INFO:
            non-existing path in 'numpy/distutils': 'site.cfg'
            INFO: lapack_opt_info:
            INFO: lapack_armpl_info:
            INFO:   libraries armpl_lp64_mp not found in ['/data/data/com.termux/files/usr/lib']
            INFO:   NOT AVAILABLE
            INFO:
            INFO: lapack_mkl_info:
            INFO:   libraries mkl_rt not found in ['/data/data/com.termux/files/usr/lib']
            INFO:   NOT AVAILABLE
            INFO:
            INFO: openblas_lapack_info:
            INFO: C compiler: aarch64-linux-android-clang -DNDEBUG -g -fwrapv -O3 -Wall -fstack-protector-strong -O3 -fstack-protector-strong -O3 -fPIC
      
            creating /data/data/com.termux/files/usr/tmp/tmpcwzgabaj/data
            creating /data/data/com.termux/files/usr/tmp/tmpcwzgabaj/data/data
            creating /data/data/com.termux/files/usr/tmp/tmpcwzgabaj/data/data/com.termux
            creating /data/data/com.termux/files/usr/tmp/tmpcwzgabaj/data/data/com.termux/files
            creating /data/data/com.termux/files/usr/tmp/tmpcwzgabaj/data/data/com.termux/files/usr
            creating /data/data/com.termux/files/usr/tmp/tmpcwzgabaj/data/data/com.termux/files/usr/tmp
            creating /data/data/com.termux/files/usr/tmp/tmpcwzgabaj/data/data/com.termux/files/usr/tmp/tmpcwzgabaj
            INFO: compile options: '-c'
            INFO: aarch64-linux-android-clang: /data/data/com.termux/files/usr/tmp/tmpcwzgabaj/source.c
            INFO: aarch64-linux-android-clang /data/data/com.termux/files/usr/tmp/tmpcwzgabaj/data/data/com.termux/files/usr/tmp/tmpcwzgabaj/source.o -L/data/data/com.termux/files/usr/lib -lopenblas -o /data/data/com.termux/files/usr/tmp/tmpcwzgabaj/a.out
            INFO:   FOUND:
            INFO:     libraries = ['openblas', 'openblas']
            INFO:     library_dirs = ['/data/data/com.termux/files/usr/lib']
            INFO:     language = c
            INFO:     define_macros = [('HAVE_CBLAS', None)]
            INFO:
            INFO:   FOUND:
            INFO:     libraries = ['openblas', 'openblas']
            INFO:     library_dirs = ['/data/data/com.termux/files/usr/lib']
            INFO:     language = c
            INFO:     define_macros = [('HAVE_CBLAS', None)]
            INFO:
            Warning: attempted relative import with no known parent package
            /data/data/com.termux/files/usr/tmp/pip-build-env-6wr647_8/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py:275: UserWarning: Unknown distribution option: 'define_macros'
              warnings.warn(msg)
            running bdist_wheel
            running build
            running config_cc
            INFO: unifing config_cc, config, build_clib, build_ext, build commands --compiler options
            running config_fc
            INFO: unifing config_fc, config, build_clib, build_ext, build commands --fcompiler options
            running build_src
            INFO: build_src
            INFO: building py_modules sources
            creating build
            creating build/src.linux-aarch64-3.11
            creating build/src.linux-aarch64-3.11/numpy
            creating build/src.linux-aarch64-3.11/numpy/distutils
            INFO: building library "npymath" sources
            WARN: Could not locate executable armflang
            WARN: Could not locate executable gfortran
            WARN: Could not locate executable f95
            WARN: Could not locate executable ifort
            WARN: Could not locate executable ifc
            WARN: Could not locate executable lf95
            WARN: Could not locate executable pgfortran
            WARN: Could not locate executable nvfortran
            WARN: Could not locate executable f90
            WARN: Could not locate executable f77
            WARN: Could not locate executable fort
            WARN: Could not locate executable efort
            WARN: Could not locate executable efc
            WARN: Could not locate executable g77
            WARN: Could not locate executable g95
            WARN: Could not locate executable pathf95
            WARN: Could not locate executable nagfor
            WARN: Could not locate executable frt
            WARN: don't know how to compile Fortran code on platform 'posix'
            creating build/src.linux-aarch64-3.11/numpy/core
            creating build/src.linux-aarch64-3.11/numpy/core/src
            creating build/src.linux-aarch64-3.11/numpy/core/src/npymath
            INFO: conv_template:> build/src.linux-aarch64-3.11/numpy/core/src/npymath/npy_math_internal.h
            INFO:   adding 'build/src.linux-aarch64-3.11/numpy/core/src/npymath' to include_dirs.
            INFO: conv_template:> build/src.linux-aarch64-3.11/numpy/core/src/npymath/ieee754.c
            INFO: conv_template:> build/src.linux-aarch64-3.11/numpy/core/src/npymath/npy_math_complex.c
            INFO: None - nothing done with h_files = ['build/src.linux-aarch64-3.11/numpy/core/src/npymath/npy_math_internal.h']
            INFO: building library "npyrandom" sources
            INFO: building extension "numpy.core._multiarray_tests" sources
            creating build/src.linux-aarch64-3.11/numpy/core/src/multiarray
            INFO: conv_template:> build/src.linux-aarch64-3.11/numpy/core/src/multiarray/_multiarray_tests.c
            INFO: building extension "numpy.core._multiarray_umath" sources
            Traceback (most recent call last):
              File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 351, in <module>
                main()
              File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 333, in main
                json_out['return_val'] = hook(**hook_input['kwargs'])
                                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 249, in build_wheel
                return _build_backend().build_wheel(wheel_directory, config_settings,
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/data/data/com.termux/files/usr/tmp/pip-build-env-6wr647_8/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 230, in build_wheel
                return self._build_with_temp_dir(['bdist_wheel'], '.whl',
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/data/data/com.termux/files/usr/tmp/pip-build-env-6wr647_8/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 215, in _build_with_temp_dir
                self.run_setup()
              File "/data/data/com.termux/files/usr/tmp/pip-build-env-6wr647_8/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 268, in run_setup
                self).run_setup(setup_script=setup_script)
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/data/data/com.termux/files/usr/tmp/pip-build-env-6wr647_8/overlay/lib/python3.11/site-packages/setuptools/build_meta.py", line 158, in run_setup
                exec(compile(code, __file__, 'exec'), locals())
              File "setup.py", line 493, in <module>
                setup_package()
              File "setup.py", line 485, in setup_package
                setup(**metadata)
              File "/data/data/com.termux/files/usr/tmp/pip-install-9wuees5h/numpy_722fd38d5586485da370ece59f23d559/numpy/distutils/core.py", line 169, in setup
                return old_setup(**new_attr)
                       ^^^^^^^^^^^^^^^^^^^^^
              File "/data/data/com.termux/files/usr/tmp/pip-build-env-6wr647_8/overlay/lib/python3.11/site-packages/setuptools/__init__.py", line 153, in setup
                return distutils.core.setup(**attrs)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/data/data/com.termux/files/usr/tmp/pip-build-env-6wr647_8/overlay/lib/python3.11/site-packages/setuptools/_distutils/core.py", line 148, in setup
                dist.run_commands()
              File "/data/data/com.termux/files/usr/tmp/pip-build-env-6wr647_8/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands
                self.run_command(cmd)
              File "/data/data/com.termux/files/usr/tmp/pip-build-env-6wr647_8/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
                cmd_obj.run()
              File "/data/data/com.termux/files/usr/tmp/pip-build-env-6wr647_8/overlay/lib/python3.11/site-packages/wheel/bdist_wheel.py", line 299, in run
                self.run_command('build')
              File "/data/data/com.termux/files/usr/tmp/pip-build-env-6wr647_8/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
                self.distribution.run_command(command)
              File "/data/data/com.termux/files/usr/tmp/pip-build-env-6wr647_8/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
                cmd_obj.run()
              File "/data/data/com.termux/files/usr/tmp/pip-install-9wuees5h/numpy_722fd38d5586485da370ece59f23d559/numpy/distutils/command/build.py", line 62, in run
                old_build.run(self)
              File "/data/data/com.termux/files/usr/tmp/pip-build-env-6wr647_8/overlay/lib/python3.11/site-packages/setuptools/_distutils/command/build.py", line 135, in run
                self.run_command(cmd_name)
              File "/data/data/com.termux/files/usr/tmp/pip-build-env-6wr647_8/overlay/lib/python3.11/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
                self.distribution.run_command(command)
              File "/data/data/com.termux/files/usr/tmp/pip-build-env-6wr647_8/overlay/lib/python3.11/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
                cmd_obj.run()
              File "/data/data/com.termux/files/usr/tmp/pip-install-9wuees5h/numpy_722fd38d5586485da370ece59f23d559/numpy/distutils/command/build_src.py", line 144, in run
                self.build_sources()
              File "/data/data/com.termux/files/usr/tmp/pip-install-9wuees5h/numpy_722fd38d5586485da370ece59f23d559/numpy/distutils/command/build_src.py", line 161, in build_sources
                self.build_extension_sources(ext)
              File "/data/data/com.termux/files/usr/tmp/pip-install-9wuees5h/numpy_722fd38d5586485da370ece59f23d559/numpy/distutils/command/build_src.py", line 318, in build_extension_sources
                sources = self.generate_sources(sources, ext)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/data/data/com.termux/files/usr/tmp/pip-install-9wuees5h/numpy_722fd38d5586485da370ece59f23d559/numpy/distutils/command/build_src.py", line 378, in generate_sources
                source = func(extension, build_dir)
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^
              File "/data/data/com.termux/files/usr/tmp/pip-install-9wuees5h/numpy_722fd38d5586485da370ece59f23d559/numpy/core/setup.py", line 513, in generate_config_h
                check_math_capabilities(config_cmd, ext, moredefs, mathlibs)
              File "/data/data/com.termux/files/usr/tmp/pip-install-9wuees5h/numpy_722fd38d5586485da370ece59f23d559/numpy/core/setup.py", line 176, in check_math_capabilities
                raise SystemError("One of the required function to build numpy is not"
            SystemError: One of the required function to build numpy is not available (the list is ['sin', 'cos', 'tan', 'sinh', 'cosh', 'tanh', 'fabs', 'floor', 'ceil', 'sqrt', 'log10', 'log', 'exp', 'asin', 'acos', 'atan', 'fmod', 'modf', 'frexp', 'ldexp']).
            [end of output]
      
        note: This error originates from a subprocess, and is likely not a problem with pip.
        ERROR: Failed building wheel for numpy
      Failed to build numpy
      ERROR: Could not build wheels for numpy, which is required to install pyproject.toml-based projects
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× pip subprocess to install build dependencies did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
$

What is the expected behavior?

It should install successfully.

System information

termux-info:

Termux Variables:
TERMUX_APP_PACKAGE_MANAGER=apt
TERMUX_MAIN_PACKAGE_FORMAT=debian
TERMUX_VERSION=0.117
Packages CPU architecture:
aarch64
Subscribed repositories:
# sources.list
deb https://cdn.lumito.net/termux/termux-main stable main
Updatable packages:
All packages up to date
termux-tools version:
1.34.1
Android version:
10
Kernel build information:
Linux localhost 4.4.205-22915538 #1 SMP PREEMPT Wed Apr 13 12:45:56 KST 2022 aarch64 Android
Device manufacturer:
samsung
Device model:
SM-T830
@tigran123 tigran123 added bug report Something is not working properly. untriaged labels Dec 24, 2022
@xtkoba
Copy link
Contributor

xtkoba commented Dec 25, 2022

First I would like to clarify that is not due to our python-numpy package. pip install scipy tries to build numpy on its own. See my comment in #12430 (comment).

@xtkoba xtkoba added python Issue is about python-related stuff, including pip. and removed untriaged labels Dec 25, 2022
@xtkoba xtkoba changed the title [Bug]: Can't install scipy and PyFFTW because of python-numpy problems. [Bug]: Can't install scipy and PyFFTW because of numpy problems. Dec 25, 2022
@tigran123
Copy link
Author

tigran123 commented Dec 25, 2022 •
edited

Yes, that is what I thought too, but I assumed that there must be something missing in python-numpy which causes both scipy and PyFFTW to try building numpy on their own. This may be a false assumption, but it at least ought to be verified, don't you think?

@tigran123
Copy link
Author

I tried both options recommended in #12430 and they don't fix the problem:

$ pip install --no-use-pep517 scipy
Collecting scipy
  Using cached scipy-1.9.3.tar.gz (42.1 MB)
ERROR: Disabling PEP 517 processing is invalid: project specifies a build backend of mesonpy in pyproject.toml
$ pip install --no-build-isolation scipy
Collecting scipy
  Using cached scipy-1.9.3.tar.gz (42.1 MB)
  Preparing metadata (pyproject.toml) ... done
ERROR: Exception:
Traceback (most recent call last):
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_internal/cli/base_command.py", line 160, in exc_logging_wrapper
    status = run_func(*args)
             ^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
    return func(self, options, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_internal/commands/install.py", line 400, in run
    requirement_set = resolver.resolve(
                      ^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 92, in resolve
    result = self._result = resolver.resolve(
                            ^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 481, in resolve
    state = resolution.resolve(requirements, max_rounds=max_rounds)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 348, in resolve
    self._add_to_criteria(self.state.criteria, r, parent=None)
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_vendor/resolvelib/resolvers.py", line 172, in _add_to_criteria
    if not criterion.candidates:
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_vendor/resolvelib/structs.py", line 151, in __bool__
    return bool(self._sequence)
           ^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 155, in __bool__
    return any(self)
           ^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 143, in <genexpr>
    return (c for c in iterator if id(c) not in self._incompatible_ids)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/found_candidates.py", line 47, in _iter_built
    candidate = func()
                ^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 206, in _make_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
                                       ^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 297, in __init__
    super().__init__(
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 162, in __init__
    self.dist = self._prepare()
                ^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 231, in _prepare
    dist = self._prepare_distribution()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 308, in _prepare_distribution
    return preparer.prepare_linked_requirement(self._ireq, parallel_builds=True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 491, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 577, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_internal/operations/prepare.py", line 69, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_internal/distributions/sdist.py", line 61, in prepare_distribution_metadata
    self.req.prepare_metadata()
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_internal/req/req_install.py", line 535, in prepare_metadata
    self.metadata_directory = generate_metadata(
                              ^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_internal/operations/build/metadata.py", line 35, in generate_metadata
    distinfo_dir = backend.prepare_metadata_for_build_wheel(metadata_dir)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_internal/utils/misc.py", line 706, in prepare_metadata_for_build_wheel
    return super().prepare_metadata_for_build_wheel(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_vendor/pep517/wrappers.py", line 189, in prepare_metadata_for_build_wheel
    return self._call_hook('prepare_metadata_for_build_wheel', {
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_vendor/pep517/wrappers.py", line 319, in _call_hook
    raise BackendUnavailable(data.get('traceback', ''))
pip._vendor.pep517.wrappers.BackendUnavailable: Traceback (most recent call last):
  File "/data/data/com.termux/files/usr/lib/python3.11/site-packages/pip/_vendor/pep517/in_process/_in_process.py", line 77, in _build_backend
    obj = import_module(mod_path)
          ^^^^^^^^^^^^^^^^^^^^^^^
  File "/data/data/com.termux/files/usr/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1206, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1178, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1142, in _find_and_load_unlocked
ModuleNotFoundError: No module named 'mesonpy'

$ pip install mesonpy
Collecting mesonpy
  Downloading MesonPy-0.0.1.tar.gz (14 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [7 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/data/data/com.termux/files/usr/tmp/pip-install-a_o6jcur/mesonpy_c8c45ca2e6334c87886e2a757efdc6dc/setup.py", line 4, in <module>
          with open('requirements.txt') as f:
               ^^^^^^^^^^^^^^^^^^^^^^^^
      FileNotFoundError: [Errno 2] No such file or directory: 'requirements.txt'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

@xtkoba
Copy link
Contributor

xtkoba commented Dec 25, 2022

I strongly believe that to be a false assumption. It would be easily verified by trying to install scipy through pip from source on any GNU/Linux system. Not that our python-numpy package is perfect and complete though.

@licy183
Copy link
Member

licy183 commented Dec 25, 2022 

wget https://github.com/scipy/scipy/releases/download/v1.9.3/scipy-1.9.3.tar.gz
tar xf scipy-1.9.3.tar.gz
cd scipy-1.9.3
python -m pip install Cython pybind11 pythran wheel
LDFLAGS="-lpython3.11" python setup.py bdist_wheel
python -m pip install ./dist/*.whl

Q&A

  1. Why not use mesonpy? mesonpy depends on ninja and cmake from scikit-build and it is well-known that they are not usable on Termux.
  2. Error 'numpy' must be installed before running the build.. This will happen on Android 7 because clogl doesn't provide by libm until Android 8. I will fix this error later.

PS: I want to provide a index repository like https://github.com/bartbroere/pypi.bartbroe.re to provide pre-built wheels for Termux in TUR, but I don't have enough time these days because I'm working on my final exams...

@xtkoba
Copy link
Contributor

xtkoba commented Dec 26, 2022

With that workaround I assume this issue can be closed.

If this is intended to be an implicit package request, then we already have one (#471) which should be used instead. I don't want discussions to be scattered across issues. Thanks.

@xtkoba xtkoba closed this as completed Dec 26, 2022
@tigran123
Copy link
Author

The above workaround fails with the error "no Fortran compiler found", so presumably I need to install lfortran package first?

@tigran123
Copy link
Author

No, even after installing lfortran it still fails like this:

running build_clib
INFO: customize UnixCCompiler
WARN: #### ['aarch64-linux-android-clang', '-DNDEBUG', '-g', '-fwrapv', '-O3', '-Wall', '-fstack-protector-strong', '-O3', '-fstack-protector-strong', '-O3'] #######
INFO: customize UnixCCompiler using build_clib
INFO: CCompilerOpt.__init__[813] : load cache from file -> /data/data/com.termux/files/home/scipy-1.9.3/build/temp.linux-aarch64-cpython-311/ccompiler_opt_cache_clib.py
INFO: CCompilerOpt.__init__[824] : hit the file cache
INFO: building 'mach' library
error: library mach has Fortran sources but no Fortran compiler found
INFO: 
########### CLIB COMPILER OPTIMIZATION ###########
INFO: Platform      : 
  Architecture: aarch64
  Compiler    : clang

CPU baseline  : 
  Requested   : 'min'
  Enabled     : NEON NEON_FP16 NEON_VFPV4 ASIMD
  Flags       : none
  Extra checks: none

CPU dispatch  : 
  Requested   : 'max -xop -fma4'
  Enabled     : ASIMDHP ASIMDDP ASIMDFHM
  Generated   : none
INFO: CCompilerOpt.cache_flush[857] : write cache to path -> /data/data/com.termux/files/home/scipy-1.9.3/build/temp.linux-aarch64-cpython-311/ccompiler_opt_cache_clib.py

@licy183
Copy link
Member

licy183 commented Dec 28, 2022

No. lfortran and flang in the main repository are not usable for compiling scipy. lfortran is under development and flang lacks libpgmath functionality. As an initial packager of flang, I don't enable libpgmath functionality because it is not available in llvm-15 and if you want to enable it a large number of cherry-pick patches (at least 16 IIRC) are needed to apply to llvm-15. So you must install gfortran from TUR or its-pointless repository.

@tigran123
Copy link
Author

Ok, so the prerequisite for your workaround is gfortran, thank you. But will adding TUR repository not conflict with python 3.11 in the main repository? I thought it had an older version of python 3.

@xtkoba xtkoba mentioned this issue Jan 21, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug report Something is not working properly. python Issue is about python-related stuff, including pip.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@tigran123 @licy183 @xtkoba