Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ARROW-16233: [Python] [Packaging] test_zoneinfo_tzinfo_to_string fails with zoneinfo._common.ZoneInfoNotFoundError on packaging wheels on Windows #12930

Closed
wants to merge 8 commits into from

Conversation

AlenkaF
Copy link
Member

@AlenkaF AlenkaF commented Apr 20, 2022

Trying to fix failing nightly-packaging jobs for windows wheels:

@github-actions
Copy link

@github-actions
Copy link

⚠️ Ticket has not been started in JIRA, please click 'Start Progress'.

@AlenkaF
Copy link
Member Author

AlenkaF commented Apr 20, 2022

@github-actions crossbow submit wheel-windows*

@github-actions
Copy link

Revision: 9541000

Submitted crossbow builds: ursacomputing/crossbow @ actions-1879

Task Status
wheel-windows-cp310-amd64 Github Actions
wheel-windows-cp37-amd64 Github Actions
wheel-windows-cp38-amd64 Github Actions
wheel-windows-cp39-amd64 Github Actions

@AlenkaF
Copy link
Member Author

AlenkaF commented Apr 20, 2022

@github-actions crossbow submit wheel-windows*

Copy link
Member

@jorisvandenbossche jorisvandenbossche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for taking a look at this!

Also in the hypothesis tests we have a try/except around importing zoneinfo, where we might additionally need to check for tzdata?

@@ -83,7 +83,7 @@ RUN choco install -r -y --no-progress python --version=%PYTHON_VERSION%
RUN python -m pip install -U pip setuptools

COPY python/requirements-wheel-build.txt arrow/python/
RUN python -m pip install -r arrow/python/requirements-wheel-build.txt
RUN python -m pip install tzdata -r arrow/python/requirements-wheel-build.txt
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the tzdata should be installed in the env for testing the wheel, and shouldn't be required when building.
We do have a python_wheel_windows_test.bat script which I suppose is relevant here.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And checking the log output, you can indeed already see that the above did not work to install tzdata in the test, because the skip is correctly working: https://github.com/ursacomputing/crossbow/runs/6089209704?check_suite_focus=true#step:7:441

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great, thanks for the pointers! I wasn't sure where to put it =)

@github-actions
Copy link

Revision: c41670f

Submitted crossbow builds: ursacomputing/crossbow @ actions-1880

Task Status
wheel-windows-cp310-amd64 Github Actions
wheel-windows-cp37-amd64 Github Actions
wheel-windows-cp38-amd64 Github Actions
wheel-windows-cp39-amd64 Github Actions

@AlenkaF
Copy link
Member Author

AlenkaF commented Apr 20, 2022

Will take a look at hypothesis tests also! 👍

@AlenkaF
Copy link
Member Author

AlenkaF commented Apr 20, 2022

@github-actions crossbow submit test-conda-python-3.8-hypothesis

@AlenkaF
Copy link
Member Author

AlenkaF commented Apr 20, 2022

@github-actions crossbow submit wheel-windows*

@github-actions
Copy link

Revision: 2c77087

Submitted crossbow builds: ursacomputing/crossbow @ actions-1881

Task Status
test-conda-python-3.8-hypothesis Github Actions

@github-actions
Copy link

Revision: 2c77087

Submitted crossbow builds: ursacomputing/crossbow @ actions-1882

Task Status
wheel-windows-cp310-amd64 Github Actions
wheel-windows-cp37-amd64 Github Actions
wheel-windows-cp38-amd64 Github Actions
wheel-windows-cp39-amd64 Github Actions

@AlenkaF AlenkaF marked this pull request as ready for review April 20, 2022 07:23
@AlenkaF
Copy link
Member Author

AlenkaF commented Apr 20, 2022

@github-actions crossbow submit test-conda-python-3.8-hypothesis

@AlenkaF
Copy link
Member Author

AlenkaF commented Apr 20, 2022

@github-actions crossbow submit wheel-windows*

@github-actions
Copy link

Revision: ca4c9b6

Submitted crossbow builds: ursacomputing/crossbow @ actions-1883

Task Status
test-conda-python-3.8-hypothesis Github Actions

@github-actions
Copy link

Failed to push updated references, potentially because of credential issues: ['refs/heads/actions-1883-github-wheel-windows-cp310-amd64', 'refs/tags/actions-1883-github-wheel-windows-cp310-amd64', 'refs/heads/actions-1883-github-wheel-windows-cp39-amd64', 'refs/tags/actions-1883-github-wheel-windows-cp39-amd64', 'refs/heads/actions-1883-github-wheel-windows-cp37-amd64', 'refs/tags/actions-1883-github-wheel-windows-cp37-amd64', 'refs/heads/actions-1883-github-wheel-windows-cp38-amd64', 'refs/tags/actions-1883-github-wheel-windows-cp38-amd64', 'refs/heads/actions-1883']
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/2195038965```

@kszucs
Copy link
Member

kszucs commented Apr 20, 2022

@github-actions crossbow submit wheel-windows*

@github-actions
Copy link

Revision: ca4c9b6

Submitted crossbow builds: ursacomputing/crossbow @ actions-1884

Task Status
wheel-windows-cp310-amd64 Github Actions
wheel-windows-cp37-amd64 Github Actions
wheel-windows-cp38-amd64 Github Actions
wheel-windows-cp39-amd64 Github Actions

@kszucs
Copy link
Member

kszucs commented Apr 20, 2022

Failed to push updated references, potentially because of credential issues: ['refs/heads/actions-1883-github-wheel-windows-cp310-amd64', 'refs/tags/actions-1883-github-wheel-windows-cp310-amd64', 'refs/heads/actions-1883-github-wheel-windows-cp39-amd64', 'refs/tags/actions-1883-github-wheel-windows-cp39-amd64', 'refs/heads/actions-1883-github-wheel-windows-cp37-amd64', 'refs/tags/actions-1883-github-wheel-windows-cp37-amd64', 'refs/heads/actions-1883-github-wheel-windows-cp38-amd64', 'refs/tags/actions-1883-github-wheel-windows-cp38-amd64', 'refs/heads/actions-1883']
The Archery job run can be found at: https://github.com/apache/arrow/actions/runs/2195038965```

This is probably caused by a race condition, another submission gets that auto incremented build ID earlier. We should remove the auto incremented IDs entirely and use unique hashes instead, e.g.: actions-<short-hash>-github-wheel-windows-cp310-amd64 instead of actions-1883-github-wheel-windows-cp310-amd64. Then we wouldn't need to fetch the new references either, making remote crossbow builds and local submission much quicker.

fyi @raulcd @assignUser

@AlenkaF
Copy link
Member Author

AlenkaF commented Apr 20, 2022

Thanks @kszucs!

Linter error is triggered because tzdata is not used directly in the code (but needs to be installed on Windows).

Copy link
Member

@jorisvandenbossche jorisvandenbossche left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me!

@jorisvandenbossche
Copy link
Member

Just one thing, a linting error .. /arrow/python/pyarrow/tests/strategies.py:35:9: F401 'tzdata' imported but unused. You will probably need to add a # noqa behind the tzdata import

@ursabot
Copy link

ursabot commented Apr 23, 2022

Benchmark runs are scheduled for baseline = 1dccb56 and contender = 36a7f11. 36a7f11 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Failed] test-mac-arm
[Failed ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.63% ⬆️0.04%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/568| 36a7f114 ec2-t3-xlarge-us-east-2>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/556| 36a7f114 test-mac-arm>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/554| 36a7f114 ursa-i9-9960x>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/566| 36a7f114 ursa-thinkcentre-m75q>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ec2-t3-xlarge-us-east-2/builds/567| 1dccb56a ec2-t3-xlarge-us-east-2>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-test-mac-arm/builds/555| 1dccb56a test-mac-arm>
[Failed] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-i9-9960x/builds/553| 1dccb56a ursa-i9-9960x>
[Finished] <https://buildkite.com/apache-arrow/arrow-bci-benchmark-on-ursa-thinkcentre-m75q/builds/565| 1dccb56a ursa-thinkcentre-m75q>
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants