Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ARROW-15504: [Python][CI] Ensure that optional components are tested #12301

Closed
wants to merge 1 commit into from
Closed

ARROW-15504: [Python][CI] Ensure that optional components are tested #12301

wants to merge 1 commit into from

Conversation

pitrou
Copy link
Member

@pitrou pitrou commented Jan 31, 2022

Tests for optional components could be silently skipped if by accident the component would fail importing despite being built.

@pitrou
Copy link
Member Author

pitrou commented Jan 31, 2022

@github-actions crossbow submit -g python -g wheel

@github-actions
Copy link

https://issues.apache.org/jira/browse/ARROW-15504

@github-actions
Copy link

Revision: 9a38be5

Submitted crossbow builds: ursacomputing/crossbow @ actions-1550

Task Status
test-conda-python-3.10 Github Actions
test-conda-python-3.7 Github Actions
test-conda-python-3.7-hdfs-2.9.2 Github Actions
test-conda-python-3.7-hdfs-3.2.1 Github Actions
test-conda-python-3.7-kartothek-latest Github Actions
test-conda-python-3.7-kartothek-master Github Actions
test-conda-python-3.7-pandas-0.24 Github Actions
test-conda-python-3.7-pandas-latest Github Actions
test-conda-python-3.7-spark-v3.1.2 Github Actions
test-conda-python-3.8 Github Actions
test-conda-python-3.8-hypothesis Github Actions
test-conda-python-3.8-pandas-latest Github Actions
test-conda-python-3.8-pandas-nightly Github Actions
test-conda-python-3.8-spark-v3.2.0 Github Actions
test-conda-python-3.9 Github Actions
test-conda-python-3.9-dask-latest Github Actions
test-conda-python-3.9-dask-master Github Actions
test-conda-python-3.9-pandas-master Github Actions
test-conda-python-3.9-spark-master Github Actions
test-debian-11-python-3 Azure
test-fedora-33-python-3 Azure
test-ubuntu-20.04-python-3 Azure
wheel-macos-big-sur-cp310-arm64 Github Actions
wheel-macos-big-sur-cp310-universal2 Github Actions
wheel-macos-big-sur-cp38-arm64 Github Actions
wheel-macos-big-sur-cp39-arm64 Github Actions
wheel-macos-big-sur-cp39-universal2 Github Actions
wheel-macos-high-sierra-cp310-amd64 Github Actions
wheel-macos-high-sierra-cp37-amd64 Github Actions
wheel-macos-high-sierra-cp38-amd64 Github Actions
wheel-macos-high-sierra-cp39-amd64 Github Actions
wheel-macos-mavericks-cp310-amd64 Github Actions
wheel-macos-mavericks-cp37-amd64 Github Actions
wheel-macos-mavericks-cp38-amd64 Github Actions
wheel-macos-mavericks-cp39-amd64 Github Actions
wheel-manylinux2010-cp310-amd64 Github Actions
wheel-manylinux2010-cp37-amd64 Github Actions
wheel-manylinux2010-cp38-amd64 Github Actions
wheel-manylinux2010-cp39-amd64 Github Actions
wheel-manylinux2014-cp310-amd64 Github Actions
wheel-manylinux2014-cp310-arm64 TravisCI
wheel-manylinux2014-cp37-amd64 Github Actions
wheel-manylinux2014-cp37-arm64 TravisCI
wheel-manylinux2014-cp38-amd64 Github Actions
wheel-manylinux2014-cp38-arm64 TravisCI
wheel-manylinux2014-cp39-amd64 Github Actions
wheel-manylinux2014-cp39-arm64 TravisCI
wheel-windows-cp310-amd64 Github Actions
wheel-windows-cp37-amd64 Github Actions
wheel-windows-cp38-amd64 Github Actions
wheel-windows-cp39-amd64 Github Actions

@pitrou pitrou force-pushed the ARROW-15504-py-test-optional-components branch from 9a38be5 to 6470f17 Compare January 31, 2022 15:07
kszucs
kszucs reviewed Jan 31, 2022
View reviewed changes
ci/scripts/python_test.sh Outdated Show resolved Hide resolved
@pitrou
ARROW-15504: [Python][CI] Ensure that optional components are tested
0838890 
Tests for optional components could be silently skipped if by accident the component would fail importing despite being built.
@pitrou pitrou force-pushed the ARROW-15504-py-test-optional-components branch from 6470f17 to 0838890 Compare January 31, 2022 16:57
@pitrou
Copy link
Member Author

pitrou commented Jan 31, 2022

@github-actions crossbow submit -g python -g wheel

@github-actions
Copy link

Revision: 0838890

Submitted crossbow builds: ursacomputing/crossbow @ actions-1554

Task Status
test-conda-python-3.10 Github Actions
test-conda-python-3.7 Github Actions
test-conda-python-3.7-hdfs-2.9.2 Github Actions
test-conda-python-3.7-hdfs-3.2.1 Github Actions
test-conda-python-3.7-kartothek-latest Github Actions
test-conda-python-3.7-kartothek-master Github Actions
test-conda-python-3.7-pandas-0.24 Github Actions
test-conda-python-3.7-pandas-latest Github Actions
test-conda-python-3.7-spark-v3.1.2 Github Actions
test-conda-python-3.8 Github Actions
test-conda-python-3.8-hypothesis Github Actions
test-conda-python-3.8-pandas-latest Github Actions
test-conda-python-3.8-pandas-nightly Github Actions
test-conda-python-3.8-spark-v3.2.0 Github Actions
test-conda-python-3.9 Github Actions
test-conda-python-3.9-dask-latest Github Actions
test-conda-python-3.9-dask-master Github Actions
test-conda-python-3.9-pandas-master Github Actions
test-conda-python-3.9-spark-master Github Actions
test-debian-11-python-3 Azure
test-fedora-33-python-3 Azure
test-ubuntu-20.04-python-3 Azure
wheel-macos-big-sur-cp310-arm64 Github Actions
wheel-macos-big-sur-cp310-universal2 Github Actions
wheel-macos-big-sur-cp38-arm64 Github Actions
wheel-macos-big-sur-cp39-arm64 Github Actions
wheel-macos-big-sur-cp39-universal2 Github Actions
wheel-macos-high-sierra-cp310-amd64 Github Actions
wheel-macos-high-sierra-cp37-amd64 Github Actions
wheel-macos-high-sierra-cp38-amd64 Github Actions
wheel-macos-high-sierra-cp39-amd64 Github Actions
wheel-macos-mavericks-cp310-amd64 Github Actions
wheel-macos-mavericks-cp37-amd64 Github Actions
wheel-macos-mavericks-cp38-amd64 Github Actions
wheel-macos-mavericks-cp39-amd64 Github Actions
wheel-manylinux2010-cp310-amd64 Github Actions
wheel-manylinux2010-cp37-amd64 Github Actions
wheel-manylinux2010-cp38-amd64 Github Actions
wheel-manylinux2010-cp39-amd64 Github Actions
wheel-manylinux2014-cp310-amd64 Github Actions
wheel-manylinux2014-cp310-arm64 TravisCI
wheel-manylinux2014-cp37-amd64 Github Actions
wheel-manylinux2014-cp37-arm64 TravisCI
wheel-manylinux2014-cp38-amd64 Github Actions
wheel-manylinux2014-cp38-arm64 TravisCI
wheel-manylinux2014-cp39-amd64 Github Actions
wheel-manylinux2014-cp39-arm64 TravisCI
wheel-windows-cp310-amd64 Github Actions
wheel-windows-cp37-amd64 Github Actions
wheel-windows-cp38-amd64 Github Actions
wheel-windows-cp39-amd64 Github Actions

@pitrou
Copy link
Member Author

pitrou commented Jan 31, 2022

@kszucs Do we lack macOS CI resources on x86-64? See Crossbow jobs above.

@pitrou pitrou closed this in 56e270f Feb 1, 2022
@pitrou pitrou deleted the ARROW-15504-py-test-optional-components branch February 1, 2022 13:13
@ursabot
Copy link

ursabot commented Feb 1, 2022 •
edited

Benchmark runs are scheduled for baseline = bf0ee3f and contender = 56e270f. 56e270f is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️1.43% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.04% ⬆️0.0%] ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

@kszucs
Copy link
Member

kszucs commented Feb 1, 2022

@kszucs Do we lack macOS CI resources on x86-64? See Crossbow jobs above.

GHA limits the number of parallel macOS runners, see https://docs.github.com/en/actions/learn-github-actions/usage-limits-billing-and-administration#usage-limits

@pitrou
Copy link
Member Author

pitrou commented Feb 1, 2022

@kszucs Do we lack macOS CI resources on x86-64? See Crossbow jobs above.

GHA limits the number of parallel macOS runners, see https://docs.github.com/en/actions/learn-github-actions/usage-limits-billing-and-administration#usage-limits

I mean we seem to have our own M1 macOS runners, so we could have some for x86-64 macOS too.

@kszucs
Copy link
Member

kszucs commented Feb 1, 2022

Sure, we can configure self-hosted runners.

@ursabot
Copy link

ursabot commented Feb 1, 2022 •
edited

Benchmark runs are scheduled for baseline = bf0ee3f and contender = 56e270f. 56e270f is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished ⬇️0.0% ⬆️0.0%] ec2-t3-xlarge-us-east-2
[Finished ⬇️0.0% ⬆️0.0%] ursa-i9-9960x
[Finished ⬇️0.48% ⬆️0.0%] ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python. Runs only benchmarks with cloud = True
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java

vibhatha pushed a commit to vibhatha/arrow that referenced this pull request Feb 4, 2022
@pitrou @vibhatha
ARROW-15504: [Python][CI] Ensure that optional components are tested
91e9653 
Tests for optional components could be silently skipped if by accident the component would fail importing despite being built.

Closes apache#12301 from pitrou/ARROW-15504-py-test-optional-components

Authored-by: Antoine Pitrou <antoine@python.org>
Signed-off-by: Antoine Pitrou <antoine@python.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@kszucs kszucs kszucs left review comments

Assignees
No one assigned
Labels
Component: Python
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
3 participants
@pitrou @ursabot @kszucs