Unable to install binary or build from master on ARM64 Ubuntu 20.04.2 LTS

[theschles]

@hmaarrfk
Still seeing problems with wheel aarch64 on master.

• Building on Ubuntu 20.04.2 LTS / focal
• Linux odroid 3.16.85-65 #1 SMP PREEMPT Mon Jan 18 13:32:38 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
• python 3.8.5

Needed scikit-image for hplip 3.21.2, and that install was failing.

So after a bit of digging and manually installing the dependencies of scikit-image 0.18.1, I think I found to manually run pip3 install scikit-image==0.18.1 --no-deps -v, 0.18.1 appeared to have a numpy conflict:

 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
  opencv-python 4.5.1.48 requires numpy>=1.19.3, but you have numpy 1.17.3 which is incompatible.
  Successfully installed Cython-0.29.23 numpy-1.17.3 setuptools-51.0.0 wheel-0.36.2
  Removed build tracker: '/tmp/pip-req-tracker-wu1ktof0'
  Installing build dependencies ... done
  Running command /usr/bin/python3 /usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpl06xbri0
  Getting requirements to build wheel ... error

Ok, lemme clone master and try...

My first run of pip install -r requirements.txt did find some dependencies that were needed. Second run shows everything in theory taken care of:

➜ pip install -r requirements.txt
Requirement already satisfied: numpy>=1.16.5 in /usr/local/lib/python3.8/dist-packages (from -r requirements/default.txt (line 1)) (1.19.3)
Requirement already satisfied: scipy>=1.2.3 in /usr/local/lib/python3.8/dist-packages (from -r requirements/default.txt (line 2)) (1.6.2)
Requirement already satisfied: matplotlib>=3.0.3 in /usr/local/lib/python3.8/dist-packages (from -r requirements/default.txt (line 3)) (3.4.1)
Requirement already satisfied: networkx>=2.2 in /usr/local/lib/python3.8/dist-packages (from -r requirements/default.txt (line 4)) (2.5.1)
Requirement already satisfied: pillow!=7.1.0,!=7.1.1,>=5.4.0 in /usr/lib/python3/dist-packages (from -r requirements/default.txt (line 5)) (7.0.0)
Requirement already satisfied: imageio>=2.4.1 in /usr/local/lib/python3.8/dist-packages (from -r requirements/default.txt (line 6)) (2.9.0)
Requirement already satisfied: tifffile>=2019.7.26 in /usr/local/lib/python3.8/dist-packages (from -r requirements/default.txt (line 7)) (2021.4.8)
Requirement already satisfied: PyWavelets>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from -r requirements/default.txt (line 8)) (1.1.1)
Requirement already satisfied: pytest>=5.2.0 in /usr/local/lib/python3.8/dist-packages (from -r requirements/test.txt (line 1)) (6.2.3)
Requirement already satisfied: pytest-cov>=2.7.0 in /usr/local/lib/python3.8/dist-packages (from -r requirements/test.txt (line 2)) (2.11.1)
Requirement already satisfied: pytest-localserver in /usr/local/lib/python3.8/dist-packages (from -r requirements/test.txt (line 3)) (0.5.0)
Requirement already satisfied: pytest-faulthandler in /usr/local/lib/python3.8/dist-packages (from -r requirements/test.txt (line 4)) (2.0.1)
Requirement already satisfied: flake8 in /usr/local/lib/python3.8/dist-packages (from -r requirements/test.txt (line 5)) (3.9.1)
Requirement already satisfied: codecov in /usr/local/lib/python3.8/dist-packages (from -r requirements/test.txt (line 6)) (2.1.11)
Requirement already satisfied: pooch>=1.3.0 in /usr/local/lib/python3.8/dist-packages (from -r requirements/test.txt (line 7)) (1.3.0)
Requirement already satisfied: pyparsing>=2.2.1 in /usr/lib/python3/dist-packages (from matplotlib>=3.0.3->-r requirements/default.txt (line 3)) (2.4.6)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.8/dist-packages (from matplotlib>=3.0.3->-r requirements/default.txt (line 3)) (0.10.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.8/dist-packages (from matplotlib>=3.0.3->-r requirements/default.txt (line 3)) (1.3.1)
Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.8/dist-packages (from matplotlib>=3.0.3->-r requirements/default.txt (line 3)) (2.8.1)
Requirement already satisfied: decorator<5,>=4.3 in /usr/local/lib/python3.8/dist-packages (from networkx>=2.2->-r requirements/default.txt (line 4)) (4.4.2)
Requirement already satisfied: packaging in /usr/lib/python3/dist-packages (from pooch>=1.3.0->-r requirements/test.txt (line 7)) (20.3)
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from pooch>=1.3.0->-r requirements/test.txt (line 7)) (2.22.0)
Requirement already satisfied: appdirs in /usr/local/lib/python3.8/dist-packages (from pooch>=1.3.0->-r requirements/test.txt (line 7)) (1.4.4)
Requirement already satisfied: iniconfig in /usr/local/lib/python3.8/dist-packages (from pytest>=5.2.0->-r requirements/test.txt (line 1)) (1.1.1)
Requirement already satisfied: toml in /usr/local/lib/python3.8/dist-packages (from pytest>=5.2.0->-r requirements/test.txt (line 1)) (0.10.2)
Requirement already satisfied: py>=1.8.2 in /usr/local/lib/python3.8/dist-packages (from pytest>=5.2.0->-r requirements/test.txt (line 1)) (1.10.0)
Requirement already satisfied: pluggy<1.0.0a1,>=0.12 in /usr/local/lib/python3.8/dist-packages (from pytest>=5.2.0->-r requirements/test.txt (line 1)) (0.13.1)
Requirement already satisfied: attrs>=19.2.0 in /usr/local/lib/python3.8/dist-packages (from pytest>=5.2.0->-r requirements/test.txt (line 1)) (20.3.0)
Requirement already satisfied: coverage>=5.2.1 in /usr/local/lib/python3.8/dist-packages (from pytest-cov>=2.7.0->-r requirements/test.txt (line 2)) (5.5)
Requirement already satisfied: six in /usr/lib/python3/dist-packages (from cycler>=0.10->matplotlib>=3.0.3->-r requirements/default.txt (line 3)) (1.14.0)
Requirement already satisfied: mccabe<0.7.0,>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from flake8->-r requirements/test.txt (line 5)) (0.6.1)
Requirement already satisfied: pycodestyle<2.8.0,>=2.7.0 in /usr/local/lib/python3.8/dist-packages (from flake8->-r requirements/test.txt (line 5)) (2.7.0)
Requirement already satisfied: pyflakes<2.4.0,>=2.3.0 in /usr/local/lib/python3.8/dist-packages (from flake8->-r requirements/test.txt (line 5)) (2.3.1)
Requirement already satisfied: werkzeug>=0.10 in /usr/local/lib/python3.8/dist-packages (from pytest-localserver->-r requirements/test.txt (line 3)) (1.0.1)

But when running pip install . it fails:

...
Installing build dependencies ... done
  Running command /usr/bin/python3 /usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpydy2wr6y
  Getting requirements to build wheel ... error
WARNING: Discarding file:///root/download/scikit-image. Command errored out with exit status -4: /usr/bin/python3 /usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpydy2wr6y Check the logs for full command output.
ERROR: Command errored out with exit status -4: /usr/bin/python3 /usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpydy2wr6y Check the logs for full command output.
Exception information:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/cli/base_command.py", line 189, in _main
    status = self.run(options, args)
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/cli/req_command.py", line 178, in wrapper
    return func(self, options, args)
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/commands/install.py", line 316, in run
    requirement_set = resolver.resolve(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/resolution/resolvelib/resolver.py", line 100, in resolve
    r = self.factory.make_requirement_from_install_req(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/resolution/resolvelib/factory.py", line 316, in make_requirement_from_install_req
    raise self._build_failures[ireq.link]
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/resolution/resolvelib/factory.py", line 167, in _make_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 300, in __init__
    super().__init__(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 144, in __init__
    self.dist = self._prepare()
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 226, in _prepare
    dist = self._prepare_distribution()
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 311, in _prepare_distribution
    return self._factory.preparer.prepare_linked_requirement(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/operations/prepare.py", line 457, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/operations/prepare.py", line 500, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/operations/prepare.py", line 66, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(finder, build_isolation)
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/distributions/sdist.py", line 39, in prepare_distribution_metadata
    self._setup_isolation(finder)
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/distributions/sdist.py", line 97, in _setup_isolation
    reqs = backend.get_requires_for_build_wheel()
  File "/usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/wrappers.py", line 177, in get_requires_for_build_wheel
    return self._call_hook('get_requires_for_build_wheel', {
  File "/usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/wrappers.py", line 274, in _call_hook
    self._subprocess_runner(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/utils/subprocess.py", line 289, in runner
    call_subprocess(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/utils/subprocess.py", line 258, in call_subprocess
    raise InstallationSubprocessError(proc.returncode, command_desc)
pip._internal.exceptions.InstallationSubprocessError: Command errored out with exit status -4: /usr/bin/python3 /usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpydy2wr6y Check the logs for full command output.
Removed file:///root/download/scikit-image from build tracker '/tmp/pip-req-tracker-y3o4iw3h'
Removed build tracker: '/tmp/pip-req-tracker-y3o4iw3h'

Full log of the install step:
scikit-image-build-fail.2021-04-18.txt

Help?

Originally posted by @theschles in #5299 (comment)

[theschles]

@hmaarrfk's response:
#5299 (comment)

My response:
#5299 (comment)

@hmaarrfk requested the output of pip list:
foo2.txt

[hmaarrfk]

It seems that pip is finding dependencies in your system libraries.

Please try to create a virtual environment, as described in:
https://scikit-image.org/docs/stable/install.html#install-via-pip

Or use conda (with conda-forge): https://github.com/conda-forge/miniforge

Let me know the results when you've created a virtual environment. The reason this is important is that pip will check for the dependencies in the virtual environment, and if it gets confused by the system libraries, it won't work well. The system libraries may be quite old and we can't control them (without getting Ubuntu involved).

[hmaarrfk]

Since you are using system dependencies, can you confirm that

sudo apt install python3-skimage

doesn't work. We've been using with the debian packager for quite a while, and they had the package working on aarch64 for quite some time.

[jni]

See this comment for an update on available wheels

[rfezzani]

I assume that #5299 fixed this issue. @theschles please feel free to reopen it if you are still unable to install skimage on ARM64 😉