Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ARROW-16908: [Python][CI] Avoid installing wrong numpy version required for testing wheels #13449

Merged
merged 2 commits into from Jun 30, 2022
Merged

ARROW-16908: [Python][CI] Avoid installing wrong numpy version required for testing wheels #13449

merged 2 commits into from Jun 30, 2022

Conversation

raulcd
Copy link
Member

@raulcd raulcd commented Jun 28, 2022

No description provided.

@github-actions
Copy link

https://issues.apache.org/jira/browse/ARROW-16908

@raulcd
Copy link
Member Author

raulcd commented Jun 28, 2022

@github-actions crossbow submit wheel-*-amd64

@github-actions
Copy link

Revision: 3b5ce7b

Submitted crossbow builds: ursacomputing/crossbow @ actions-0a19738808

Task Status
wheel-macos-high-sierra-cp310-amd64 Github Actions
wheel-macos-high-sierra-cp37-amd64 Github Actions
wheel-macos-high-sierra-cp38-amd64 Github Actions
wheel-macos-high-sierra-cp39-amd64 Github Actions
wheel-macos-mavericks-cp310-amd64 Github Actions
wheel-macos-mavericks-cp37-amd64 Github Actions
wheel-macos-mavericks-cp38-amd64 Github Actions
wheel-macos-mavericks-cp39-amd64 Github Actions
wheel-manylinux2010-cp310-amd64 Github Actions
wheel-manylinux2010-cp37-amd64 Github Actions
wheel-manylinux2010-cp38-amd64 Github Actions
wheel-manylinux2010-cp39-amd64 Github Actions
wheel-manylinux2014-cp310-amd64 Github Actions
wheel-manylinux2014-cp37-amd64 Github Actions
wheel-manylinux2014-cp38-amd64 Github Actions
wheel-manylinux2014-cp39-amd64 Github Actions
wheel-windows-cp310-amd64 Github Actions
wheel-windows-cp37-amd64 Github Actions
wheel-windows-cp38-amd64 Github Actions
wheel-windows-cp39-amd64 Github Actions

@raulcd raulcd marked this pull request as ready for review June 28, 2022 18:00
@raulcd
Copy link
Member Author

raulcd commented Jun 28, 2022

@jorisvandenbossche this PR fixes the current wheel failures on the nightly packaging jobs. What do you think? This issue introduced on https://github.com/apache/arrow/pull/12763/files#diff-a2c3d2d0c3a36128695e156b63504e1dd91f4167f23afe4fa8bb6e131ef248c1

@kou
Copy link
Member

kou commented Jun 28, 2022

It seems that https://github.com/ursacomputing/crossbow/runs/7091332365?check_suite_focus=true#step:7:315 causes the problem:

+ pip install --force-reinstall /arrow/python/repaired_wheels/pyarrow-9.0.0.dev276-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Processing /arrow/python/repaired_wheels/pyarrow-9.0.0.dev276-cp310-cp310-manylinux_2_12_x86_64.manylinux2010_x86_64.whl
Collecting numpy>=1.16.6
  Downloading numpy-1.23.0-cp310-cp310-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (17.0 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 17.0/17.0 MB 116.5 MB/s eta 0:00:00
Installing collected packages: numpy, pyarrow
  Attempting uninstall: numpy
    Found existing installation: numpy 1.21.3
    Uninstalling numpy-1.21.3:
      Successfully uninstalled numpy-1.21.3
Successfully installed numpy-1.23.0 pyarrow-9.0.0.dev276

Why does pip install pyarrow.whl upgrade installed NumPy to 1.23.0 from 1.21.3 for numpy>=1.16.6 condition?

@raulcd
Copy link
Member Author

raulcd commented Jun 29, 2022

Why does pip install pyarrow.whl upgrade installed NumPy to 1.23.0 from 1.21.3 for numpy>=1.16.6 condition?

The problem is the --force-reinstall which:

--force-reinstall
    Reinstall all packages even if they are already up-to-date.

I am removing it and pushing to CI to test all the wheels. It worked locally so I am not sure why the --force-reinstall was required as I can see the local pyarrow wheel being installed.

@raulcd
Copy link
Member Author

raulcd commented Jun 29, 2022

@github-actions crossbow submit wheel-*

@github-actions
Copy link

Revision: 7b84094

Submitted crossbow builds: ursacomputing/crossbow @ actions-995d8c2366

Task Status
wheel-macos-big-sur-cp310-arm64 Github Actions
wheel-macos-big-sur-cp310-universal2 Github Actions
wheel-macos-big-sur-cp38-arm64 Github Actions
wheel-macos-big-sur-cp39-arm64 Github Actions
wheel-macos-big-sur-cp39-universal2 Github Actions
wheel-macos-high-sierra-cp310-amd64 Github Actions
wheel-macos-high-sierra-cp37-amd64 Github Actions
wheel-macos-high-sierra-cp38-amd64 Github Actions
wheel-macos-high-sierra-cp39-amd64 Github Actions
wheel-macos-mavericks-cp310-amd64 Github Actions
wheel-macos-mavericks-cp37-amd64 Github Actions
wheel-macos-mavericks-cp38-amd64 Github Actions
wheel-macos-mavericks-cp39-amd64 Github Actions
wheel-manylinux2010-cp310-amd64 Github Actions
wheel-manylinux2010-cp37-amd64 Github Actions
wheel-manylinux2010-cp38-amd64 Github Actions
wheel-manylinux2010-cp39-amd64 Github Actions
wheel-manylinux2014-cp310-amd64 Github Actions
wheel-manylinux2014-cp310-arm64 TravisCI
wheel-manylinux2014-cp37-amd64 Github Actions
wheel-manylinux2014-cp37-arm64 TravisCI
wheel-manylinux2014-cp38-amd64 Github Actions
wheel-manylinux2014-cp38-arm64 TravisCI
wheel-manylinux2014-cp39-amd64 Github Actions
wheel-manylinux2014-cp39-arm64 TravisCI
wheel-windows-cp310-amd64 Github Actions
wheel-windows-cp37-amd64 Github Actions
wheel-windows-cp38-amd64 Github Actions
wheel-windows-cp39-amd64 Github Actions

@raulcd raulcd changed the title ARROW-16908: [Python][CI] Install required wheel test dependencies before testing wheels ARROW-16908: [Python][CI] Avoid installing wrong numpy version required for testing wheels Jun 29, 2022
@kou
Copy link
Member

kou commented Jun 29, 2022

The --force-reinstall was added by @pitrou de75bb5 in #11316 .

@pitrou Do you remember why you added the --force-reinstall?

@pitrou
Copy link
Member

pitrou commented Jun 30, 2022

@kou I don't remember. Perhaps because Numpy didn't install properly? We can remove it if it works.

kou
kou approved these changes Jun 30, 2022
View reviewed changes
Copy link
Member

@kou kou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK.

+1

@kou kou merged commit 841be69 into apache:master Jun 30, 2022
vibhatha pushed a commit to vibhatha/arrow that referenced this pull request Jul 5, 2022
@raulcd @vibhatha
ARROW-16908: [Python][CI] Avoid installing wrong numpy version requir…
c70a4c7 
…ed for testing wheels (apache#13449)

Authored-by: Raúl Cumplido <raulcumplido@gmail.com>
Signed-off-by: Sutou Kouhei <kou@clear-code.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kou kou kou approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@raulcd @kou @pitrou