Skip to content

Add minimum dependencies CI build#4794

Open
jrbourbeau wants to merge 29 commits intodask:mainfrom
jrbourbeau:mindeps-ci
Open

Add minimum dependencies CI build#4794
jrbourbeau wants to merge 29 commits intodask:mainfrom
jrbourbeau:mindeps-ci

Conversation

@jrbourbeau
Copy link
Member

This PR adds a new CI build which runs the test suite against the minimum version of distributeds dependencies (similar to what we already do over in dask/dask). This will allow us to test, to the best of our ability, that our specified minimum versions are valid.

Comment on lines +1 to +8
click >= 6.7
cloudpickle >= 1.5.0
dask >= 2021.03.0
dask >= 2021.04.1
msgpack >= 0.6.0
psutil >= 5.0
sortedcontainers !=2.0.0, !=2.0.1
psutil >= 5.4.7
sortedcontainers >= 2.0.4
tblib >= 1.6.0
toolz >= 0.8.2
toolz >= 0.10.0
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that:

  • The minimum versions for click, psutil, and sortedcontainers were bumped to fix conda solve errors

  • dask was bumped to avoid the issue reported here 2021.04.1 release community#150 (comment)

  • toolz itself didn't need to be bumped, however there was a segmentation fault in CI due to cytoolz (which is pulled in from the dask conda package). Bumping the minimum toolz version indirectly leads to a correspondingly newer version of cytoolz being installed. Given that (cy)toolz=0.10.0 is almost two years old, I think this is okay

@jrbourbeau jrbourbeau changed the title [WIP] Add minimum dependencies CI build Add minimum dependencies CI build May 7, 2021
@jrbourbeau jrbourbeau mentioned this pull request May 11, 2021
3 tasks
@jrbourbeau jrbourbeau self-assigned this Nov 18, 2021
@jrbourbeau
Copy link
Member Author

@crusaderky would you mind taking a look at this PR when you get a chance?

@crusaderky
Copy link
Collaborator

crusaderky commented Dec 16, 2021

I think you should really have two separate tests:

  1. test that everything works when pinning minimum version of various dependencies
  2. test that everything works when you exclusively have the listed minimum dependencies (e.g. no numpy, pandas, bokeh)

ATM you're trying to do both in one test and as a result you're skipping a lot of tests - which significantly harms test coverage vs. minimum versions.

@fjetter fjetter added enhancement Improve existing functionality or make things work better tests Unit tests and/or continuous integration labels May 25, 2022
@jrbourbeau
Copy link
Member Author

Circling back here, it'd be good to get a minimum dependency CI build into distributed. This would have helped us avoid the need for our recent 2022.10.2 hotfix release (xref dask/community#284). @charlesbluca this seems like the sort of thing you might enjoy -- do you have bandwidth / interest in getting this across the the finish line?

@charlesbluca
Copy link
Member

Thanks for the ping @jrbourbeau! Yeah I can look into finishing this up

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement Improve existing functionality or make things work better tests Unit tests and/or continuous integration

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants