Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Python][CI] Dask integration tests are failing for tests that should be skipped with pyarrow strings #37307

Closed
jorisvandenbossche opened this issue Aug 22, 2023 · 1 comment · Fixed by #37324
Closed

[Python][CI] Dask integration tests are failing for tests that should be skipped with pyarrow strings #37307

jorisvandenbossche opened this issue Aug 22, 2023 · 1 comment · Fixed by #37324
Assignees
@jorisvandenbossche
Labels
Component: Continuous Integration Component: Python
Milestone
14.0.0

Comments

@jorisvandenbossche
Copy link
Member

For example https://github.com/ursacomputing/crossbow/actions/runs/5932877661/job/16087448914. Three tests are failing:

 FAILED opt/conda/envs/arrow/lib/python3.11/site-packages/dask/dataframe/tests/test_dataframe.py::test_meta_nonempty_uses_meta_value_if_provided
FAILED opt/conda/envs/arrow/lib/python3.11/site-packages/dask/dataframe/tests/test_dataframe.py::test_dataframe_explode
FAILED opt/conda/envs/arrow/lib/python3.11/site-packages/dask/dataframe/tests/test_dataframe.py::test_series_explode

If I run those locally, I get the message that those tests are skipped:

dask/dataframe/tests/test_dataframe.py::test_series_explode SKIPPED (No need to run with pyarrow strings)

This seems to be correct. But for some reason, on our nightly CI, the @pytest.mark.skip_with_pyarrow_strings is not working correctly
@jorisvandenbossche
Copy link
Member Author

Related to pytest-dev/pytest#1596 I assume

jorisvandenbossche added a commit to jorisvandenbossche/arrow that referenced this issue Aug 23, 2023
@jorisvandenbossche
apacheGH-37307: [Python][CI] Manually skip tests with skip_with_pyarr…
393ad32 
…ow_strings marker for nightly dask integration tests
@github-actions github-actions bot mentioned this issue Aug 23, 2023
Merged
jorisvandenbossche added a commit that referenced this issue Aug 24, 2023
@jorisvandenbossche
GH-37307: [Python][CI] Manually skip tests with skip_with_pyarrow_str…
c927520 
…ings marker for nightly dask integration tests (#37324)

### Rationale for this change

Dask added some tests with a custom `@ pytest.mark.skip_with_pyarrow_strings` mark, which ensures to skip some tests when pyarrow is installed. However, that skip doesn't work correctly when running the tests on an installed version of dask with `pytest --pyargs dask.tests`. 
Therefore manually skipping tests with that mark as a workaround.

* Closes: #37307

Authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
@jorisvandenbossche jorisvandenbossche added this to the 14.0.0 milestone Aug 24, 2023
loicalleyne pushed a commit to loicalleyne/arrow that referenced this issue Nov 13, 2023
@jorisvandenbossche @loicalleyne
apacheGH-37307: [Python][CI] Manually skip tests with skip_with_pyarr…
793a476 
…ow_strings marker for nightly dask integration tests (apache#37324)

### Rationale for this change

Dask added some tests with a custom `@ pytest.mark.skip_with_pyarrow_strings` mark, which ensures to skip some tests when pyarrow is installed. However, that skip doesn't work correctly when running the tests on an installed version of dask with `pytest --pyargs dask.tests`. 
Therefore manually skipping tests with that mark as a workaround.

* Closes: apache#37307

Authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jorisvandenbossche jorisvandenbossche

Labels
Component: Continuous Integration Component: Python
Projects
None yet
Milestone
14.0.0
1 participant
@jorisvandenbossche