Dedicated .py file instead of snippets in extraConfig

[albertmichaelj]

I'm opening this issue primarily for advice, but I also think that there may have been an unintentional side effect of a recent change to helm chart. #1407 moved the config files for the hub image to the helm chart files/ directory. Overall, this seems to be a good thing, but there seems to be a problem with trying to use a custom hub image with a custom configuration with this helm chart as a sub-chart. Maybe I'm just misunderstanding something.

Before this change, if I wanted to change jupyterhub_config.py, all I needed to do was build a custom hub image with my new jupyterhub_config.py file and then point the hub image in the helm chart to my custom image. Now though, even with a custom docker image for the hub, the helm chart version of jupyterhub_config.py gets mounted and run. Moreover, if I'm using this helm chart as a sub-chart in a larger helm app, I can't figure out a way I could overwrite the jupyterhub_config.py file without just forking this chart entirely and modifying the jupyterhub_config.py file directly.

Am I missing something? Did the hub customization become harder when using this chart as a sub-chart?

I'm very open to ideas, and just to be clear, using the extra_config is not enough because I want to sub-class KubeSpawner.

[consideRatio]

I'm very open to ideas, and just to be clear, using the extra_config is not enough because I want to sub-class KubeSpawner.

I think you can actually sub class kubespawner from the extraConfig. Consider it as a python file that will run. You can define the sub class and then assign it. Something like...

hub:
  extraConfig:
    customSpawner: |
      import kubespawner
      class MySpawner(kubespawner.KubeSpawner):
          pass
      c.JupyterHub.spawner_class = 'MySpawner'

If you really want to replace everything, then hmm... Perhaps override the command to start jupyterhub to use the specified configuration that you build with the image could accomplish it?

[albertmichaelj]

Your right that I certainly could sub class KubeSpawner in extra_config, but I think I would basically have to redo any part of the config file that sets values for the c.KubeSpawner... stuff, right? I would certainly have to do that as well if I had my own jupyterhub_config.py file, but it would be easier to do a find and replace instead of maintaining most of the jupyterhub_config.py as stuff in extra_config. Or maybe I'm misunderstanding how the c.KubeSpawner... stuff works.

As for overriding the command, how would you do that? The command is not part of the values.yaml. It seems to be hardcoded into the template according to this line. Is it possible to override the command to start jupyterhub when you are using this chart as a subchart?

[consideRatio]

Ah, no I actually think you would be good to go even though c.KubeSpawner.config_options are set and you wont need to reset these.

Regarding the hub command, woops... Okay so it is hardcoded, and then it is hard to change. We could certainly make this not hardcoded. It would make sense to me.

[albertmichaelj]

If it wasn't hardcoded, then this would be a path forward. Maybe that is the best that can be done right now. That should be a pretty minor change, right?

However, what would be really great is if we could somehow override the entire jupyterhub_config.py file with our own file. I have been trying to figure out some way to override files in subcharts, but there doesn't seem to be a way to currently do that. If we could do something like this, then we could also do away with the extra_config key and just add another file called jupyterhub_custom_config.py that gets run from the juptyerhub_config.py. This would make maintaining the extra config a lot nicer since we wouldn't have to maintain python code in a yaml file.

I did find a reference to the idea of using library charts in Helm 3 to do something similar to this, but to be honest, my understanding of helm is too limited to have fully parsed what was going on.

[consideRatio]

Hi again @albertmichaelj, I think it is quite unique to want to override the entire jupyterhub_config.py, because you would need to know a lot and abandon the entire configuration system of the Helm chart pretty much.

If you make a good case for wanting to replace the entire jupyterhub_config.py file, then let's add the option to configure the hub command or similarly. If so, the approach would be exactly like in #1813, I just confirmed that JupyterHub would also replace the earlier flags with the latter flags.

[albertmichaelj]

@consideRatio Thanks for following up! My use case was because I wanted to include significant amount of code in the JupyterHub config file. Specifically, I had intended to write a custom authenticator in the JuptyerHub config file that would have been a large amount of code. Trying to maintain this code as part of the helm chart configuration file would have been a nightmare. To be clear, my use case would have involved modifying the existing configuration file instead of starting from scratch (which I completely agree is unrealistic in any situation I can think of).

My solution to this was to instead create my own hub image and a package for my authenticator that I could just install in the hub image. I honestly believe that for my use case, creating my own hub image was probably the correct approach. However, when I opened this issue, I think I was in a bit of an uncanny valley of competency with kubernetes, so I believed that editing the configuration file would be a simpler solution. So, I do not believe that my use case requires an editable configuration file.

However, I still am inclined to think that having an option to swap out the config file is still a better approach than hardcoding it. It's hard to anticipate how someone might want to modify the configuration (while I won't go into it now, I am doing things with JupyterHub on Kubernetes that were clearly not the intention of this chart). However, I understand that additional configurability implies an on-going maintenance burden, so this may not be worth the effort.

Feel free to close this if you think that is appropriate.

[consideRatio]

@albertmichaelj thank you for your thorough follow up to this!

It sounds to me like you wanted to avoid the hub.extraConfig issue of embedding python in YAML as a big string, and instead be able to use a dedicated .py file somhoe, for example to make use of syntax highlighting and such. Perhaps that can be accomplished some other way still. Does this represent the bigger portion of your issue with using hub.extraConfig?

[albertmichaelj]

My issue was exactly that hub.extraConfig is not a great way to add significant amounts of code to the configuration file. It is a very convenient mechanism for small snippets of code that you need to add, but it doesn't work great for things like defining new authentication classes. So, I think in most cases if you could just load a second .py file somehow things would be fine, especially given that you could likely overwrite any default values that the original config file set. It seems like there are probably edge cases where it would still be better to have a different config file, but I can't actually imagine what those edge cases would be.

[adamd01]

@albertmichaelj - I had the same exact issue on upgrading

I settled on this as a workaround in the end:

• Add custom code into a file e.g. jupyterhub_config_custom.py
• Add the file as a config map kubectl create configmap --namespace ${JUPYTER_NAMESPACE} jupyterhub-config --from-file jupyterhub_config_custom.py --dry-run -o yaml | kubectl apply -f -
• Mount the config map as a volume and have hub.extraConfig read that file and exec it

hub:
  extraVolumeMounts:
    - name: jupyterhub-config
      mountPath: /etc/jupyterhub/jupyterhub_config_custom.py
      subPath: jupyterhub_config_custom.py
  extraVolumes:
    - name: jupyterhub-config
      configMap:
        name: jupyterhub-config
  extraConfig:
    01-custom-config: |
      config_py = open('/etc/jupyterhub/jupyterhub_config_custom.py').read()
      exec(config_py)

The original jupyterhub_config.py still runs first, but that shouldn't really be an issue I don't think

[consideRatio]

I'm closing this in favor of the issue I mentioned this issue from (see reference just above this comment). It includes some thoughts on how to go about it and such.