Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Version 0.18.2 -- aarch64 wheels #5299

Closed
4 tasks done
hmaarrfk opened this issue Mar 30, 2021 · 26 comments
Closed
4 tasks done

Version 0.18.2 -- aarch64 wheels #5299

hmaarrfk opened this issue Mar 30, 2021 · 26 comments
Labels
⏩ type: Enhancement Improve existing features 🤖 type: Infrastructure CI, packaging, tools and automation

Comments

@hmaarrfk
Copy link
Member

hmaarrfk commented Mar 30, 2021 •
edited

On the road to building aarch64 wheels, we are trying to release 0.18.2

Presently, from what I know, we need to:

cc: @janaknat
@hmaarrfk
Copy link
Member Author

I'm trying to build 32 bit wheels with verbose testing in:
https://github.com/hmaarrfk/scikit-image/actions/runs/699799160

@grlee77 grlee77 added ⏩ type: Enhancement Improve existing features 🤖 type: Infrastructure CI, packaging, tools and automation labels Mar 31, 2021
@grlee77
Copy link
Contributor

grlee77 commented Mar 31, 2021

It looks like the failure occurs within a call to np.eigvalsh when NumPy calls the LAPACK function DSYEVD:
https://github.com/hmaarrfk/scikit-image/runs/2223694716?check_suite_focus=true#step:6:6498

  =================================== FAILURES ===================================
  ____________________ test_multiscale_basic_features_channel ____________________
  
      def test_multiscale_basic_features_channel():
          img = np.zeros((10, 10, 5))
          img[:10] = 1
          img += 0.05 * np.random.randn(*img.shape)
          n_sigmas = 2
          features = multiscale_basic_features(img, sigma_min=1, sigma_max=2, multichannel=True)
          assert features.shape[-1] == 5 * n_sigmas * 4
          assert features.shape[:-1] == img.shape[:-1]
          # Consider last axis as spatial dimension
  >       features = multiscale_basic_features(img, sigma_min=1, sigma_max=2)
  
  /tmp/tmp.xmdLqkZ753/venv/lib/python3.9/site-packages/skimage/feature/tests/test_basic_features.py:28: 
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  /tmp/tmp.xmdLqkZ753/venv/lib/python3.9/site-packages/skimage/feature/_basic_features.py:171: in multiscale_basic_features
      features = list(itertools.chain.from_iterable(all_results))
  /tmp/tmp.xmdLqkZ753/venv/lib/python3.9/site-packages/skimage/feature/_basic_features.py:159: in <genexpr>
      _mutiscale_basic_features_singlechannel(
  /tmp/tmp.xmdLqkZ753/venv/lib/python3.9/site-packages/skimage/feature/_basic_features.py:87: in _mutiscale_basic_features_singlechannel
      out_sigmas = list(
  /opt/python/cp39-cp39/lib/python3.9/concurrent/futures/_base.py:600: in result_iterator
      yield fs.pop().result()
  /opt/python/cp39-cp39/lib/python3.9/concurrent/futures/_base.py:433: in result
      return self.__get_result()
  /opt/python/cp39-cp39/lib/python3.9/concurrent/futures/_base.py:389: in __get_result
      raise self._exception
  /opt/python/cp39-cp39/lib/python3.9/concurrent/futures/thread.py:52: in run
      result = self.fn(*self.args, **self.kwargs)
  /tmp/tmp.xmdLqkZ753/venv/lib/python3.9/site-packages/skimage/feature/_basic_features.py:89: in <lambda>
      lambda s: _singlescale_basic_features_singlechannel(
  /tmp/tmp.xmdLqkZ753/venv/lib/python3.9/site-packages/skimage/feature/_basic_features.py:28: in _singlescale_basic_features_singlechannel
      results += (*_texture_filter(gaussian_filtered),)
  /tmp/tmp.xmdLqkZ753/venv/lib/python3.9/site-packages/skimage/feature/_basic_features.py:14: in _texture_filter
      eigvals = feature.hessian_matrix_eigvals(H_elems)
  /tmp/tmp.xmdLqkZ753/venv/lib/python3.9/site-packages/skimage/feature/corner.py:413: in hessian_matrix_eigvals
      return _symmetric_compute_eigenvalues(H_elems)
  /tmp/tmp.xmdLqkZ753/venv/lib/python3.9/site-packages/skimage/feature/corner.py:275: in _symmetric_compute_eigenvalues
      eigs = np.linalg.eigvalsh(matrices)[..., ::-1]
  <__array_function__ internals>:5: in eigvalsh
      ???
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  
  a = array([[[[[-1.21414661e-04, -1.69277191e-04,  3.27229500e-04],
            [-1.69277191e-04, -6.97374344e-06, -2.1612644...[ 9.68217850e-04, -5.34951687e-05, -3.97205353e-04],
            [ 1.41447783e-03, -3.97205353e-04,  4.82678413e-04]]]]])
  UPLO = 'L'
  
      @array_function_dispatch(_eigvalsh_dispatcher)
      def eigvalsh(a, UPLO='L'):
          """
          >>> a = np.array([[1, -2j], [2j, 5]])
          >>> LA.eigvalsh(a)
          array([ 0.17157288,  5.82842712]) # may vary
      
          >>> # demonstrate the treatment of the imaginary part of the diagonal
          >>> a = np.array([[5+2j, 9-2j], [0+2j, 2-1j]])
          >>> a
          array([[5.+2.j, 9.-2.j],
                 [0.+2.j, 2.-1.j]])
          >>> # with UPLO='L' this is numerically equivalent to using LA.eigvals()
          >>> # with:
          >>> b = np.array([[5.+0.j, 0.-2.j], [0.+2.j, 2.-0.j]])
          >>> b
          array([[5.+0.j, 0.-2.j],
                 [0.+2.j, 2.+0.j]])
          >>> wa = LA.eigvalsh(a)
          >>> wb = LA.eigvals(b)
          >>> wa; wb
          array([1., 6.])
          array([6.+0.j, 1.+0.j])
      
          """
          UPLO = UPLO.upper()
          if UPLO not in ('L', 'U'):
              raise ValueError("UPLO argument must be 'L' or 'U'")
      
          extobj = get_linalg_error_extobj(
              _raise_linalgerror_eigenvalues_nonconvergence)
          if UPLO == 'L':
              gufunc = _umath_linalg.eigvalsh_lo
          else:
              gufunc = _umath_linalg.eigvalsh_up
      
          a, wrap = _makearray(a)
          _assert_stacked_2d(a)
          _assert_stacked_square(a)
          t, result_t = _commonType(a)
          signature = 'D->d' if isComplexType(t) else 'd->d'
  >       w = gufunc(a, signature=signature, extobj=extobj)
  E       ValueError: On entry to DSYEVD parameter number 8 had an illegal value

<\details>

@grlee77
Copy link
Contributor

grlee77 commented Mar 31, 2021 •
edited

Earlier in the log, pip does not find a NumPy wheel, so it gets built from source instead. It looks like there is an aarch64 wheel for Python 3.9 on PyPI, but it was built with manylinux2014: https://pypi.org/project/numpy/#files

@alexdesiqueira alexdesiqueira mentioned this issue Mar 31, 2021
Closed
@hmaarrfk
Copy link
Member Author

hmaarrfk commented Apr 1, 2021

@grlee77 i guess not only do i have to bump the normal python 3.9 requirements for x86, but i have to bump manylinux even more for aarch64.

@hmaarrfk
Copy link
Member Author

hmaarrfk commented Apr 1, 2021

It seems that the very first release of numpy for aarch was using manylinux2014

https://pypi.org/project/numpy/1.20.0/#files

I can probably pin to that.

@hmaarrfk
Copy link
Member Author

hmaarrfk commented Apr 1, 2021

It seems that 2014 is the current default for aarch64

@hmaarrfk
Copy link
Member Author

@jni not sure what is left to release 0.18.2. I think it would be beneficial to do it sooner rather than later since people have been asking for an aarch build.

Thoughts?

@hmaarrfk hmaarrfk mentioned this issue Apr 15, 2021
Merged
@theschles
Copy link

theschles commented Apr 18, 2021 •
edited

@hmaarrfk
Still seeing problems with wheel aarch64 on master.

  • Building on Ubuntu 20.04.2 LTS / focal
  • Linux odroid 3.16.85-65 #1 SMP PREEMPT Mon Jan 18 13:32:38 UTC 2021 aarch64 aarch64 aarch64 GNU/Linux
  • python 3.8.5

Needed scikit-image for hplip 3.21.2, and that install was failing.

So after a bit of digging and manually installing the dependencies of scikit-image 0.18.1, I think I found to manually run pip3 install scikit-image==0.18.1 --no-deps -v, 0.18.1 appeared to have a numpy conflict:

 ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
  opencv-python 4.5.1.48 requires numpy>=1.19.3, but you have numpy 1.17.3 which is incompatible.
  Successfully installed Cython-0.29.23 numpy-1.17.3 setuptools-51.0.0 wheel-0.36.2
  Removed build tracker: '/tmp/pip-req-tracker-wu1ktof0'
  Installing build dependencies ... done
  Running command /usr/bin/python3 /usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpl06xbri0
  Getting requirements to build wheel ... error

Ok, lemme clone master and try...

My first run of pip install -r requirements.txt did find some dependencies that were needed. Second run shows everything in theory taken care of:

➜ pip install -r requirements.txt
Requirement already satisfied: numpy>=1.16.5 in /usr/local/lib/python3.8/dist-packages (from -r requirements/default.txt (line 1)) (1.19.3)
Requirement already satisfied: scipy>=1.2.3 in /usr/local/lib/python3.8/dist-packages (from -r requirements/default.txt (line 2)) (1.6.2)
Requirement already satisfied: matplotlib>=3.0.3 in /usr/local/lib/python3.8/dist-packages (from -r requirements/default.txt (line 3)) (3.4.1)
Requirement already satisfied: networkx>=2.2 in /usr/local/lib/python3.8/dist-packages (from -r requirements/default.txt (line 4)) (2.5.1)
Requirement already satisfied: pillow!=7.1.0,!=7.1.1,>=5.4.0 in /usr/lib/python3/dist-packages (from -r requirements/default.txt (line 5)) (7.0.0)
Requirement already satisfied: imageio>=2.4.1 in /usr/local/lib/python3.8/dist-packages (from -r requirements/default.txt (line 6)) (2.9.0)
Requirement already satisfied: tifffile>=2019.7.26 in /usr/local/lib/python3.8/dist-packages (from -r requirements/default.txt (line 7)) (2021.4.8)
Requirement already satisfied: PyWavelets>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from -r requirements/default.txt (line 8)) (1.1.1)
Requirement already satisfied: pytest>=5.2.0 in /usr/local/lib/python3.8/dist-packages (from -r requirements/test.txt (line 1)) (6.2.3)
Requirement already satisfied: pytest-cov>=2.7.0 in /usr/local/lib/python3.8/dist-packages (from -r requirements/test.txt (line 2)) (2.11.1)
Requirement already satisfied: pytest-localserver in /usr/local/lib/python3.8/dist-packages (from -r requirements/test.txt (line 3)) (0.5.0)
Requirement already satisfied: pytest-faulthandler in /usr/local/lib/python3.8/dist-packages (from -r requirements/test.txt (line 4)) (2.0.1)
Requirement already satisfied: flake8 in /usr/local/lib/python3.8/dist-packages (from -r requirements/test.txt (line 5)) (3.9.1)
Requirement already satisfied: codecov in /usr/local/lib/python3.8/dist-packages (from -r requirements/test.txt (line 6)) (2.1.11)
Requirement already satisfied: pooch>=1.3.0 in /usr/local/lib/python3.8/dist-packages (from -r requirements/test.txt (line 7)) (1.3.0)
Requirement already satisfied: pyparsing>=2.2.1 in /usr/lib/python3/dist-packages (from matplotlib>=3.0.3->-r requirements/default.txt (line 3)) (2.4.6)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.8/dist-packages (from matplotlib>=3.0.3->-r requirements/default.txt (line 3)) (0.10.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.8/dist-packages (from matplotlib>=3.0.3->-r requirements/default.txt (line 3)) (1.3.1)
Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.8/dist-packages (from matplotlib>=3.0.3->-r requirements/default.txt (line 3)) (2.8.1)
Requirement already satisfied: decorator<5,>=4.3 in /usr/local/lib/python3.8/dist-packages (from networkx>=2.2->-r requirements/default.txt (line 4)) (4.4.2)
Requirement already satisfied: packaging in /usr/lib/python3/dist-packages (from pooch>=1.3.0->-r requirements/test.txt (line 7)) (20.3)
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from pooch>=1.3.0->-r requirements/test.txt (line 7)) (2.22.0)
Requirement already satisfied: appdirs in /usr/local/lib/python3.8/dist-packages (from pooch>=1.3.0->-r requirements/test.txt (line 7)) (1.4.4)
Requirement already satisfied: iniconfig in /usr/local/lib/python3.8/dist-packages (from pytest>=5.2.0->-r requirements/test.txt (line 1)) (1.1.1)
Requirement already satisfied: toml in /usr/local/lib/python3.8/dist-packages (from pytest>=5.2.0->-r requirements/test.txt (line 1)) (0.10.2)
Requirement already satisfied: py>=1.8.2 in /usr/local/lib/python3.8/dist-packages (from pytest>=5.2.0->-r requirements/test.txt (line 1)) (1.10.0)
Requirement already satisfied: pluggy<1.0.0a1,>=0.12 in /usr/local/lib/python3.8/dist-packages (from pytest>=5.2.0->-r requirements/test.txt (line 1)) (0.13.1)
Requirement already satisfied: attrs>=19.2.0 in /usr/local/lib/python3.8/dist-packages (from pytest>=5.2.0->-r requirements/test.txt (line 1)) (20.3.0)
Requirement already satisfied: coverage>=5.2.1 in /usr/local/lib/python3.8/dist-packages (from pytest-cov>=2.7.0->-r requirements/test.txt (line 2)) (5.5)
Requirement already satisfied: six in /usr/lib/python3/dist-packages (from cycler>=0.10->matplotlib>=3.0.3->-r requirements/default.txt (line 3)) (1.14.0)
Requirement already satisfied: mccabe<0.7.0,>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from flake8->-r requirements/test.txt (line 5)) (0.6.1)
Requirement already satisfied: pycodestyle<2.8.0,>=2.7.0 in /usr/local/lib/python3.8/dist-packages (from flake8->-r requirements/test.txt (line 5)) (2.7.0)
Requirement already satisfied: pyflakes<2.4.0,>=2.3.0 in /usr/local/lib/python3.8/dist-packages (from flake8->-r requirements/test.txt (line 5)) (2.3.1)
Requirement already satisfied: werkzeug>=0.10 in /usr/local/lib/python3.8/dist-packages (from pytest-localserver->-r requirements/test.txt (line 3)) (1.0.1)

But when running pip install . it fails:

...
Installing build dependencies ... done
  Running command /usr/bin/python3 /usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpydy2wr6y
  Getting requirements to build wheel ... error
WARNING: Discarding file:///root/download/scikit-image. Command errored out with exit status -4: /usr/bin/python3 /usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpydy2wr6y Check the logs for full command output.
ERROR: Command errored out with exit status -4: /usr/bin/python3 /usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpydy2wr6y Check the logs for full command output.
Exception information:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/cli/base_command.py", line 189, in _main
    status = self.run(options, args)
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/cli/req_command.py", line 178, in wrapper
    return func(self, options, args)
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/commands/install.py", line 316, in run
    requirement_set = resolver.resolve(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/resolution/resolvelib/resolver.py", line 100, in resolve
    r = self.factory.make_requirement_from_install_req(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/resolution/resolvelib/factory.py", line 316, in make_requirement_from_install_req
    raise self._build_failures[ireq.link]
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/resolution/resolvelib/factory.py", line 167, in _make_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 300, in __init__
    super().__init__(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 144, in __init__
    self.dist = self._prepare()
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 226, in _prepare
    dist = self._prepare_distribution()
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 311, in _prepare_distribution
    return self._factory.preparer.prepare_linked_requirement(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/operations/prepare.py", line 457, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/operations/prepare.py", line 500, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/operations/prepare.py", line 66, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(finder, build_isolation)
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/distributions/sdist.py", line 39, in prepare_distribution_metadata
    self._setup_isolation(finder)
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/distributions/sdist.py", line 97, in _setup_isolation
    reqs = backend.get_requires_for_build_wheel()
  File "/usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/wrappers.py", line 177, in get_requires_for_build_wheel
    return self._call_hook('get_requires_for_build_wheel', {
  File "/usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/wrappers.py", line 274, in _call_hook
    self._subprocess_runner(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/utils/subprocess.py", line 289, in runner
    call_subprocess(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/utils/subprocess.py", line 258, in call_subprocess
    raise InstallationSubprocessError(proc.returncode, command_desc)
pip._internal.exceptions.InstallationSubprocessError: Command errored out with exit status -4: /usr/bin/python3 /usr/local/lib/python3.8/dist-packages/pip/_vendor/pep517/_in_process.py get_requires_for_build_wheel /tmp/tmpydy2wr6y Check the logs for full command output.
Removed file:///root/download/scikit-image from build tracker '/tmp/pip-req-tracker-y3o4iw3h'
Removed build tracker: '/tmp/pip-req-tracker-y3o4iw3h'

Full log of the install step:
scikit-image-build-fail.2021-04-18.txt

Help?

@hmaarrfk
Copy link
Member Author

I guess the requirements.txt are not fully in sync with the pyproject.toml. can you try removing the pyproject.toml file? It lists the requirement of the lowest version of bumpy that exists on pypi as a wheel for aarch.

It seems that you have built it yourself though with a lower version. Is that correct?

@theschles
Copy link

theschles commented Apr 19, 2021 •
edited

@hmaarrfk I have not yet successfully installed scikit-image:

  • neither from a pre-built binary e.g. pip3 install scikit-image==0.18.1
  • nor from master using the steps I showed in my prior post

When I removed the pyproject.toml file, it refused to build:

root in scikit-image on  main [✘]
➜ git pull
Already up to date.

root in scikit-image on  main [✘]
➜ git status
On branch main
Your branch is up to date with 'origin/main'.

Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
	deleted:    pyproject.toml

no changes added to commit (use "git add" and/or "git commit -a")

➜ pip install -r requirements.txt
Requirement already satisfied: numpy>=1.16.5 in /usr/local/lib/python3.8/dist-packages (from -r requirements/default.txt (line 1)) (1.19.3)
Requirement already satisfied: scipy>=1.2.3 in /usr/local/lib/python3.8/dist-packages (from -r requirements/default.txt (line 2)) (1.6.2)
Requirement already satisfied: matplotlib>=3.0.3 in /usr/local/lib/python3.8/dist-packages (from -r requirements/default.txt (line 3)) (3.4.1)
Requirement already satisfied: networkx>=2.2 in /usr/local/lib/python3.8/dist-packages (from -r requirements/default.txt (line 4)) (2.5.1)
Requirement already satisfied: pillow!=7.1.0,!=7.1.1,>=5.4.0 in /usr/lib/python3/dist-packages (from -r requirements/default.txt (line 5)) (7.0.0)
Requirement already satisfied: imageio>=2.4.1 in /usr/local/lib/python3.8/dist-packages (from -r requirements/default.txt (line 6)) (2.9.0)
Requirement already satisfied: tifffile>=2019.7.26 in /usr/local/lib/python3.8/dist-packages (from -r requirements/default.txt (line 7)) (2021.4.8)
Requirement already satisfied: PyWavelets>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from -r requirements/default.txt (line 8)) (1.1.1)
Requirement already satisfied: pytest>=5.2.0 in /usr/local/lib/python3.8/dist-packages (from -r requirements/test.txt (line 1)) (6.2.3)
Requirement already satisfied: pytest-cov>=2.7.0 in /usr/local/lib/python3.8/dist-packages (from -r requirements/test.txt (line 2)) (2.11.1)
Requirement already satisfied: pytest-localserver in /usr/local/lib/python3.8/dist-packages (from -r requirements/test.txt (line 3)) (0.5.0)
Requirement already satisfied: pytest-faulthandler in /usr/local/lib/python3.8/dist-packages (from -r requirements/test.txt (line 4)) (2.0.1)
Requirement already satisfied: flake8 in /usr/local/lib/python3.8/dist-packages (from -r requirements/test.txt (line 5)) (3.9.1)
Requirement already satisfied: codecov in /usr/local/lib/python3.8/dist-packages (from -r requirements/test.txt (line 6)) (2.1.11)
Requirement already satisfied: pooch>=1.3.0 in /usr/local/lib/python3.8/dist-packages (from -r requirements/test.txt (line 7)) (1.3.0)
Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.8/dist-packages (from matplotlib>=3.0.3->-r requirements/default.txt (line 3)) (2.8.1)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.8/dist-packages (from matplotlib>=3.0.3->-r requirements/default.txt (line 3)) (1.3.1)
Requirement already satisfied: pyparsing>=2.2.1 in /usr/lib/python3/dist-packages (from matplotlib>=3.0.3->-r requirements/default.txt (line 3)) (2.4.6)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.8/dist-packages (from matplotlib>=3.0.3->-r requirements/default.txt (line 3)) (0.10.0)
Requirement already satisfied: decorator<5,>=4.3 in /usr/local/lib/python3.8/dist-packages (from networkx>=2.2->-r requirements/default.txt (line 4)) (4.4.2)
Requirement already satisfied: packaging in /usr/lib/python3/dist-packages (from pooch>=1.3.0->-r requirements/test.txt (line 7)) (20.3)
Requirement already satisfied: requests in /usr/lib/python3/dist-packages (from pooch>=1.3.0->-r requirements/test.txt (line 7)) (2.22.0)
Requirement already satisfied: appdirs in /usr/local/lib/python3.8/dist-packages (from pooch>=1.3.0->-r requirements/test.txt (line 7)) (1.4.4)
Requirement already satisfied: toml in /usr/local/lib/python3.8/dist-packages (from pytest>=5.2.0->-r requirements/test.txt (line 1)) (0.10.2)
Requirement already satisfied: py>=1.8.2 in /usr/local/lib/python3.8/dist-packages (from pytest>=5.2.0->-r requirements/test.txt (line 1)) (1.10.0)
Requirement already satisfied: iniconfig in /usr/local/lib/python3.8/dist-packages (from pytest>=5.2.0->-r requirements/test.txt (line 1)) (1.1.1)
Requirement already satisfied: pluggy<1.0.0a1,>=0.12 in /usr/local/lib/python3.8/dist-packages (from pytest>=5.2.0->-r requirements/test.txt (line 1)) (0.13.1)
Requirement already satisfied: attrs>=19.2.0 in /usr/local/lib/python3.8/dist-packages (from pytest>=5.2.0->-r requirements/test.txt (line 1)) (20.3.0)
Requirement already satisfied: coverage>=5.2.1 in /usr/local/lib/python3.8/dist-packages (from pytest-cov>=2.7.0->-r requirements/test.txt (line 2)) (5.5)
Requirement already satisfied: six in /usr/lib/python3/dist-packages (from cycler>=0.10->matplotlib>=3.0.3->-r requirements/default.txt (line 3)) (1.14.0)
Requirement already satisfied: mccabe<0.7.0,>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from flake8->-r requirements/test.txt (line 5)) (0.6.1)
Requirement already satisfied: pyflakes<2.4.0,>=2.3.0 in /usr/local/lib/python3.8/dist-packages (from flake8->-r requirements/test.txt (line 5)) (2.3.1)
Requirement already satisfied: pycodestyle<2.8.0,>=2.7.0 in /usr/local/lib/python3.8/dist-packages (from flake8->-r requirements/test.txt (line 5)) (2.7.0)
Requirement already satisfied: werkzeug>=0.10 in /usr/local/lib/python3.8/dist-packages (from pytest-localserver->-r requirements/test.txt (line 3)) (1.0.1)

root in scikit-image on  main [✘] took 4s
➜ pip3 install . -v > ../foo.txt 2>&1

root in scikit-image on  main [✘]
➜ cat ../foo.txt
Using pip 21.0.1 from /usr/local/lib/python3.8/dist-packages/pip (python 3.8)
Non-user install because site-packages writeable
Created temporary directory: /tmp/pip-ephem-wheel-cache-o269e4ky
Created temporary directory: /tmp/pip-req-tracker-naw3sf_3
Initialized build tracking at /tmp/pip-req-tracker-naw3sf_3
Created build tracker: /tmp/pip-req-tracker-naw3sf_3
Entered build tracker: /tmp/pip-req-tracker-naw3sf_3
Created temporary directory: /tmp/pip-install-dhtgqx1o
Processing /root/download/scikit-image
  Created temporary directory: /tmp/pip-req-build-kzt7kjoo
  Added file:///root/download/scikit-image to build tracker '/tmp/pip-req-tracker-naw3sf_3'
    Running setup.py (path:/tmp/pip-req-build-kzt7kjoo/setup.py) egg_info for package from file:///root/download/scikit-image
    Created temporary directory: /tmp/pip-pip-egg-info-pa27l48p
    Running command python setup.py egg_info
WARNING: Discarding file:///root/download/scikit-image. Command errored out with exit status -4: python setup.py egg_info Check the logs for full command output.
ERROR: Command errored out with exit status -4: python setup.py egg_info Check the logs for full command output.
Exception information:
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/cli/base_command.py", line 189, in _main
    status = self.run(options, args)
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/cli/req_command.py", line 178, in wrapper
    return func(self, options, args)
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/commands/install.py", line 316, in run
    requirement_set = resolver.resolve(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/resolution/resolvelib/resolver.py", line 100, in resolve
    r = self.factory.make_requirement_from_install_req(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/resolution/resolvelib/factory.py", line 316, in make_requirement_from_install_req
    raise self._build_failures[ireq.link]
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/resolution/resolvelib/factory.py", line 167, in _make_candidate_from_link
    self._link_candidate_cache[link] = LinkCandidate(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 300, in __init__
    super().__init__(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 144, in __init__
    self.dist = self._prepare()
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 226, in _prepare
    dist = self._prepare_distribution()
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 311, in _prepare_distribution
    return self._factory.preparer.prepare_linked_requirement(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/operations/prepare.py", line 457, in prepare_linked_requirement
    return self._prepare_linked_requirement(req, parallel_builds)
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/operations/prepare.py", line 500, in _prepare_linked_requirement
    dist = _get_prepared_distribution(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/operations/prepare.py", line 66, in _get_prepared_distribution
    abstract_dist.prepare_distribution_metadata(finder, build_isolation)
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/distributions/sdist.py", line 41, in prepare_distribution_metadata
    self.req.prepare_metadata()
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/req/req_install.py", line 549, in prepare_metadata
    self.metadata_directory = self._generate_metadata()
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/req/req_install.py", line 524, in _generate_metadata
    return generate_metadata_legacy(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/operations/build/metadata_legacy.py", line 70, in generate_metadata
    call_subprocess(
  File "/usr/local/lib/python3.8/dist-packages/pip/_internal/utils/subprocess.py", line 258, in call_subprocess
    raise InstallationSubprocessError(proc.returncode, command_desc)
pip._internal.exceptions.InstallationSubprocessError: Command errored out with exit status -4: python setup.py egg_info Check the logs for full command output.
Removed file:///root/download/scikit-image from build tracker '/tmp/pip-req-tracker-naw3sf_3'
Removed build tracker: '/tmp/pip-req-tracker-naw3sf_3'

root in scikit-image on  main [✘]
➜

@hmaarrfk
Copy link
Member Author

@theschles i'm really sorry to ask, but could you please open a new issue. The goal of this issue is to track progress on the 0.18.2 release which we plan to get out soon (in order to have aarch wheels).

In the issue, can you please provide the output of pip list

@jni
Copy link
Member

jni commented Apr 20, 2021

@hmaarrfk I think we can go ahead and release assuming all is green on the 0.18.x branch. Separately, @grlee77 has started working with @leej3 and @viniciusdc, also at Quansight, to overhaul our release process, so maybe this is a good chance to get them to witness the mess 😂. Perhaps we can schedule a day next week to have a big release party? ;)

@leej3
Copy link
Contributor

leej3 commented Apr 20, 2021

Sign me up!

@theschles theschles mentioned this issue Apr 20, 2021
Closed
@jni
Copy link
Member

jni commented May 6, 2021

@janaknat @theschles I've pushed 3.7 and 3.9 aarch64 wheels for 0.18.2rc1. You can try installing them with pip install -U --pre scikit-image. Unfortunately there are no 3.8 wheel because this workflow failed. I've rerun it a couple of times but it keeps failing — I'm not sure whether it's because of a bad github actions cache?

At any rate, the wheels the action did upload are built on manylinux2014 for arch. I can't see which part of the config sets that — it seems to be implicitly the only architecture it can use, because the env is set to manylinux2010.

Anyway, let us know whether these wheels work for you, if so, I'll tag 0.18.2 and push those.

The 0.18.2 release notes can be found here.

@grlee77
Copy link
Contributor

grlee77 commented May 6, 2021

That rank filter failure may be resolved by backporting #5175

@janaknat
Copy link
Contributor

janaknat commented May 6, 2021

@jni The workflow that failed was due to i686 having an error. Hopefully, with the PR suggested by @grlee77 it gets resolved. With respect to manylinux, aarch64 is only supported starting with manylinux2014. cibuildwheel automatically does the right thing for aarch64.

I can test the wheels. Any particular test you'd like to see me run to check for correctness?

@hmaarrfk
Copy link
Member Author

hmaarrfk commented May 6, 2021

I personally think we can skip hardware testing. We already release aarch stuff on conda-forge and it seems to be fine in the wild.

@jni
Copy link
Member

jni commented May 7, 2021

I can test the wheels. Any particular test you'd like to see me run to check for correctness?

Whatever it is you're using skimage for! 😂 ie if your feedback is that the wheels are fit for purpose we'll push the release — now with the latest test-skipping patch! 🎉

@geoffreyblake
Copy link

@jni, @janaknat , I gave scikit-image 0.18.2rc1 wheels a test on Arm64 with Python3.7 and Python3.9. The binary portions load fine and seem to be working. I think they are good to go with a release.

Its a shame the 3.8 wheels are not there. The i686 build may be handling the pickled arrays differently and that is why its failing with a bad magic. I found this reference: https://stackoverflow.com/questions/40054872/numpy-load-wrong-magic-string-error

Maybe explicitly telling the np.load function in filters/rank/testing/test_rank.py:61 the file should be interpreted as bytes would help? https://numpy.org/doc/stable/reference/generated/numpy.load.html

@jni
Copy link
Member

jni commented Jun 4, 2021 •
edited

@geoffreyblake thank you for the feedback! The missing wheel was in fact fixed by #5383. I just pushed a new tag, hopefully will push a new release candidate in a few hours when the builds finish. 🤞

@jni
Copy link
Member

jni commented Jun 5, 2021

Ok skimage 0.18.2rc2 is released, with aarch64 wheels for Python 3.{7,8,9}! @geoffreyblake @janaknat if you give the go-ahead with these I will release 0.18.2 proper! Apologies for the long cycle, we are getting there! 🙏

@geoffreyblake
Copy link

@jni sciki-image 0.18.2rc2 appears to be working fine on aarch64 for python 3.{7,8.9}. I believe it is ready for release.

@janaknat
Copy link
Contributor

@jni Any ideas on when the aarch64 wheels get released?

@jni
Copy link
Member

jni commented Jun 29, 2021

Ooops! I tagged the release but never uploaded the wheels and source to PyPI! This is why the work of @leej3 and @viniciusdc in #5397 is so important! 😅 Release happening right now! 🚨🚨🚨

@jni
Copy link
Member

jni commented Jun 29, 2021

Release made! So happy that I can finally close this! Thank you everyone for all your contributions and for your patience! 🙏

@jni jni closed this as completed Jun 29, 2021
@janaknat
Copy link
Contributor

janaknat commented Jul 1, 2021

Thank you @jni !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
⏩ type: Enhancement Improve existing features 🤖 type: Infrastructure CI, packaging, tools and automation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@hmaarrfk @jni @theschles @leej3 @grlee77 @geoffreyblake @janaknat