[Spike] Determine what is needed to make dask-gateway a toggleable option in basehub #4087
Assignees
Comments
|
I had an idea while showering yesterday, so am going to explore this a little bit and report back. |
yuvipanda
added a commit
to yuvipanda/pilot-hubs
that referenced
this issue
Jun 18, 2024
I have timeboxed myself to 30min, and I have verified that this
gives us a fairly seamless migration pathway for existing installations
of daskhub (no re-install needed), and allows us to mix multiple
dependencies (primarily binderhub-service and dask-gateway) more
easily. It does this by taking the following two actions:
- Make dask-gateway a conditional dependency of basehub,
rather than daskhub
- Move all remaining daskhub/values.yaml config into basehub,
either under dask-gateway, or under an extraConfig that's enabled
conditionally (via `custom.daskGateway.enabled`)
The end result here is that `daskhub` is now a stub that is
*completely empty*, and simply passes through to basehub! So basehub
is where *all* the action is, and `daskhub` is a simple backwards
compatible passthrough.
I've tested this on the openscapes staging hub and verified that
everything works as it should! I only had to make the following
three config additions:
1. Set `basehub.dask-gateway.enabled` to `true`
2. Set `basehub.jupyterhub.custom.daskGateway.enabled` to `true`.
3. Set `basehub.jupyterhub.singleuser.cloudMetadata.blockWithIptables` to
`true`. We may not need this, but I didn't want to investigate that in
this spike.
Remaining things to do to complete this migration:
- [ ] Remove *all* references to `daskhub` in our deployer code.
If something needs to know if dask-gateway is enabled, it
should check for `basehub.dask-gateway.enabled`
- [ ] Make appropriate config changes to all existing daskhub
installations so this is a seamless transition.
- [ ] Document the fact that some of our existing hub config
will have a `daskhub` moniker as a historical artifact. I
think the additional minor cognitive overhead is totally
worth the fact that we won't have to re-install any hubs!
Ref 2i2c-org#4087
|
Outcome reported in #4240 |
|
Given @GeorgianaElena's comments in #4240, I think we can mark this as done - as the goal of this spike was to find information. |
GeorgianaElena
pushed a commit
to yuvipanda/pilot-hubs
that referenced
this issue
Jun 27, 2024
I have timeboxed myself to 30min, and I have verified that this
gives us a fairly seamless migration pathway for existing installations
of daskhub (no re-install needed), and allows us to mix multiple
dependencies (primarily binderhub-service and dask-gateway) more
easily. It does this by taking the following two actions:
- Make dask-gateway a conditional dependency of basehub,
rather than daskhub
- Move all remaining daskhub/values.yaml config into basehub,
either under dask-gateway, or under an extraConfig that's enabled
conditionally (via `custom.daskGateway.enabled`)
The end result here is that `daskhub` is now a stub that is
*completely empty*, and simply passes through to basehub! So basehub
is where *all* the action is, and `daskhub` is a simple backwards
compatible passthrough.
I've tested this on the openscapes staging hub and verified that
everything works as it should! I only had to make the following
three config additions:
1. Set `basehub.dask-gateway.enabled` to `true`
2. Set `basehub.jupyterhub.custom.daskGateway.enabled` to `true`.
3. Set `basehub.jupyterhub.singleuser.cloudMetadata.blockWithIptables` to
`true`. We may not need this, but I didn't want to investigate that in
this spike.
Remaining things to do to complete this migration:
- [ ] Remove *all* references to `daskhub` in our deployer code.
If something needs to know if dask-gateway is enabled, it
should check for `basehub.dask-gateway.enabled`
- [ ] Make appropriate config changes to all existing daskhub
installations so this is a seamless transition.
- [ ] Document the fact that some of our existing hub config
will have a `daskhub` moniker as a historical artifact. I
think the additional minor cognitive overhead is totally
worth the fact that we won't have to re-install any hubs!
Ref 2i2c-org#4087
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
(Prior work in #3252).
As part of https://github.com/2i2c-org/meta/issues/1105, we want to offer the possibility of dask-gateway along with the binderhub UI. This requires us to enable both the
binderhub-servicechart and thedask-gatewaychart. We can currently do this because ourdaskhubinherits frombasehub, but it means we need to make the determination of dask-gateway or not at the very beginning. It would be useful to instead move to a fully composable model, where we can toggle this on and off as needed.This spike is to investigate what is needed for this to be done. The outcome of this would be a more detailed report in #3252
The text was updated successfully, but these errors were encountered: