Skip to content

Commit

Permalink
GH-37307: [Python][CI] Manually skip tests with skip_with_pyarrow_str…
Browse files Browse the repository at this point in the history
…ings marker for nightly dask integration tests (#37324)

### Rationale for this change

Dask added some tests with a custom `@ pytest.mark.skip_with_pyarrow_strings` mark, which ensures to skip some tests when pyarrow is installed. However, that skip doesn't work correctly when running the tests on an installed version of dask with `pytest --pyargs dask.tests`. 
Therefore manually skipping tests with that mark as a workaround.

* Closes: #37307

Authored-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
Signed-off-by: Joris Van den Bossche <jorisvandenbossche@gmail.com>
  • Loading branch information
jorisvandenbossche committed Aug 24, 2023
1 parent 9958896 commit c927520
Showing 1 changed file with 4 additions and 2 deletions.
6 changes: 4 additions & 2 deletions ci/scripts/integration_dask.sh
Original file line number Diff line number Diff line change
Expand Up @@ -31,11 +31,13 @@ python -c "import dask.dataframe"
# pytest -sv --pyargs dask.bytes.tests.test_hdfs
# pytest -sv --pyargs dask.bytes.tests.test_local

pytest -v --pyargs dask.dataframe.tests.test_dataframe
# The "skip_with_pyarrow_strings" marker is meant to skip automatically, but that doesn't work with --pyargs, so de-selecting manually
pytest -v --pyargs dask.dataframe.tests.test_dataframe -m "not skip_with_pyarrow_strings"
pytest -v --pyargs dask.dataframe.io.tests.test_orc
# skip failing parquet tests
# test_pandas_timestamp_overflow_pyarrow is skipped because of GH-33321.
pytest -v --pyargs dask.dataframe.io.tests.test_parquet \
-k "not test_pandas_timestamp_overflow_pyarrow"
-k "not test_pandas_timestamp_overflow_pyarrow" \
-m "not skip_with_pyarrow_strings and not xfail_with_pyarrow_strings"
# this file contains parquet tests that use S3 filesystem
pytest -v --pyargs dask.bytes.tests.test_s3

0 comments on commit c927520

Please sign in to comment.