Skip to content

Fix mindeps-distributed CI build to handle numpy/pandas not being installed #9668

Merged
jrbourbeau merged 2 commits intodask:mainfrom
jrbourbeau:test-ci-27
Nov 16, 2022
Merged

Fix mindeps-distributed CI build to handle numpy/pandas not being installed #9668
jrbourbeau merged 2 commits intodask:mainfrom
jrbourbeau:test-ci-27

Conversation

@jrbourbeau
Copy link
Copy Markdown
Member

No description provided.

@jrbourbeau jrbourbeau changed the title Test CI Fix mindeps-distributed CI build to handle numpy/pandas not being installed Nov 16, 2022
@jrbourbeau
Copy link
Copy Markdown
Member Author

Okay, so this is an interesting one. On main we've started getting

FAILED dask/tests/test_layers.py::test_dataframe_cull_key_dependencies[_shuffle_op] - ImportError: Dask dataframe requirements are not installed.

Please either conda or pip install as follows:

  conda install dask                     # either conda install
  python -m pip install "dask[dataframe]" --upgrade  # or python -m pip install

in our mindep-distributed build due to numpy/pandas not being installed. Looking closer at test_dataframe_cull_key_dependencies, we are in fact missing a pytest.importorskip("dask.dataframe") (which has been added in this PR). Though I was confused by we weren't seeing this before. It turns out that previously we were skipping this test (and all the test in test_layers.py) due to the following issue when attempting to import distributed:

dask/tests/test_layers.py:5: could not import 'distributed': tlz.peekn

Which was recently fixed in dask/distributed#7309. So we had been skipping this test, but for the wrong reasons -- this PR makes it so we skip/run tests appropriately

cc @charlesbluca @ncclementi in case either of you have a moment to look at this

@jrbourbeau
Copy link
Copy Markdown
Member Author

Will plan to merge this after CI finishes in order to get CI passing ASAP. Happy to handle any related changes in a follow-up PR

@jrbourbeau jrbourbeau merged commit 0dd82eb into dask:main Nov 16, 2022
@jrbourbeau jrbourbeau deleted the test-ci-27 branch November 16, 2022 19:17
Copy link
Copy Markdown
Member

@rjzamora rjzamora left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for looking into this @jrbourbeau ! CI failure seems like a known flake.

EDIT: Looks like my approval was a few seconds too late :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants