Release 2021.05.0

[jrbourbeau]

I'd like to release dask and distributed 2021.05.0 this Friday. In particular it would be nice to get dask/distributed#4810 out to users.

Additionally:

• Fix bug in Dataframe column projection dask#7645
• Merge global annotations when packing dask#7565
• Merge global annotations on the client distributed#4691

would also be good to include in the relase

cc @jakirkham @jsignell @kkraus14

[jakirkham]

Should we hot-fix the existing packages to put a lower bound on click version? Apparently these are causing people issues as well

cc @quasiben

[jrbourbeau]

Sure, pinning click<8 on existing dask and distributed conda releases seems reasonable. I'm not sure if there's anything we can do on PyPI though

[jakirkham]

Sounds good. Submitted PR ( conda-forge/conda-forge-repodata-patches-feedstock#142 ) to fix the repodata and PR ( conda-forge/distributed-feedstock#165 ) to fix the current package

Unfortunately PyPI doesn't have a concept of repodata that I know of. So aside from yanking old packages there is not much we can do

[jrbourbeau]

Following up on #150 (comment) and dask/distributed#4819, should we try pinning dask and distributed more closely with this release? That is, make dask=2021.05.0 depend on distributed=2021.05.0 (and vice versa). There's been recent churn in the codebase, particularly around developing protocols around HighLevelGraphs, and since we don't currently specify upper bounds for our dependencies there's been some user pain that pinning more tightly would help avoid.

FWIW I brought this up a recent maintainers meeting and there were no objections to pinning dask and distributed more tightly.

@jakirkham @quasiben @kkraus14 would this negatively impact RAPIDS?

EDIT: My preference is to try pinning more tightly

[jsignell]

I just saw dask/distributed#4820 and can't tell if it is an issue that needs to be solved before the release

[martindurant]

I does not sound like a new issue, but would be worth someone's while to see if it worked with older versions.

[jakirkham]

It looks like a usage issue. The code should await the compute call. That said, it should also be using client.compute instead of dask.compute (I don't think the latter can be used with async). Added a modified example for the user that does work

Edit: James pointed out this traces back to an older issue that was raised previously in 2019. In it Jim outlines what would be needed to do to improve the situation ( dask/dask#5518 (comment) ) as well as why it doesn't work today. Presumably this issue has existed as long as the async Client has

[jrbourbeau]

Re: dask/distributed#4820 I agree that's a longstanding issue, and there's certainly room for improvement, but it shouldn't be a blocker for releasing.

Re: version pinning, I spoke with @quasiben offline and he mentioned pinning dask and distributed more closely shouldn't be an issue for RAPIDS. To be clear, here are the explicit updates I'd like to make as part of the release today:

• For releases on PyPI, we'll update the setup.py files for dask and distributed such that distributed=2021.05.0 depends on dask=2021.05.0 and the dask[distributed] extras_require pins distributed=2021.05.0.

• For releases on conda-forge, we'll have distributed depend on dask-core=2021.05.0 and the dask meta-package depend on dask-core=2021.05.0 and distributed=2021.05.0.

If this goes well, then we should continue to bump dask and distributed pinnings for future releases. So do the same thing as outlined above, but for the 2021.05.1 release, 2021.06.0 release, 2021.06.1 release, and so on.

[jrbourbeau]

Alright, I'm going to starting pushing out this release

[jrbourbeau]

2021.05.0 is on PyPI, conda-forge, and Docker Hub. Thanks all!

[max-sixty]

Hi there — forgive asking on the old issue — but given the discussion above, is it also inadvisable to run say 2021.05.0 with 2021.06.0? Or is that in general OK?

[mrocklin]

In general you should try to match versions.

…

On Sat, Jun 19, 2021 at 8:34 PM Maximilian Roos ***@***.***> wrote: Hi there — forgive asking on the old issue — but given the discussion above, is it also inadvisable to run say 2021.05.0 with 2021.06.0? Or is that in general OK? — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#155 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACKZTF4BSCPHHYBT72X5GTTTVAT3ANCNFSM442VQE2Q> .