Add cores and memory fields to SpecCluster widget for scaling options

[guillaumeeb]

Following #2974, it would be nice to modify the Jupyter widget to provide cores and memory fields. This was done in dask-jobqueue.

I can work on this later if it's OK.

[mrocklin]

One challenge is that that code was SpecCluster specific, and so wouldn't work well with other systems that don't use SpecCluster. In particular, Dask-Yarn uses a different system. @jcrist any thoughts on trying to support operations like scale(memory="1TB") ?

…

On Thu, Aug 22, 2019 at 8:09 AM Guillaume Eynard-Bontemps < ***@***.***> wrote: Following #2974 <#2974>, it would be nice to modify the Jupyter widget to provide cores and memory fields. This was done in dask-jobqueue. I can work on this later if it's OK. — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#2985?email_source=notifications&email_token=AACKZTHE6Y7DXRBJ6A6JHZLQF2T3LA5CNFSM4IOWESBKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4HG2BYHA>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AACKZTGGWBQQWDAD3ROYN7LQF2T3LANCNFSM4IOWESBA> .

[mrocklin]

@jcrist , before you go out on vacation could I ask you to chime in here?

Do you have any interest in supporting operations like scale(memory="1 TB") within dask-{yarn,gateway}? If so, that would allow us to avoid splitting off the widget code as we try to support previous workloads in dask-jobqueue (it might also be a decent idea on its own).

[mrocklin]

Poke @jcrist

[guillaumeeb]

This issue has gone stale, but I'm still interested in adding these keywords in the Cluster widget.
I've not updated Dask in a while on my setup, but I don't thing this has been done?

Any news or comment @mrocklin @jcrist ?

[mrocklin]

I don't think that anything has happened here yet. I agree that it would be good to do. We might want to check that the Cluster's scale method supports these keywords, perhaps with the utils.has_keyword function.

…

On Fri, Feb 21, 2020 at 2:20 PM Guillaume Eynard-Bontemps < ***@***.***> wrote: This issue has gone stale, but I'm still interested in adding these keywords in the Cluster widget. I've not updated Dask in a while on my setup, but I don't thing this has been done? Any news or comment @mrocklin <https://github.com/mrocklin> @jcrist <https://github.com/jcrist> ? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#2985?email_source=notifications&email_token=AACKZTAT6O7EOX27DYKKPMDREBHUVA5CNFSM4IOWESBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMUJGNQ#issuecomment-589861686>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACKZTASEURQU3NX3NDJ753REBHUVANCNFSM4IOWESBA> .