Skip to content

Env variable for extra_template_basedirs #2026

@govinda18

Description

@govinda18

Hi team,

I want to be able to set extra_template_basedirs via some env variable. I am using in an internal infrastructure that needs this and some external library seems to use PythonExporter to convert a notebook into python.

Would be helpful to set this globally so this gets automatically picked up.

Activity

added a commit that references this issue on Aug 3, 2023
30af202
Carreau

Carreau commented on Aug 10, 2023

@Carreau
Member

I'd like to understand this request a bit more instead of providing a band aid for a single option.

Do you have a way of creating ~/.jupyter/jupyter_nbconvert_config.py (or any other default config file) ? You should be able to also find all the places you can add those default configs using jupyter --paths.

With this you should be able to hook into any logics you want without waiting for a patch/release.

$ cat ~/.jupyter/jupyter_nbconvert_config.py
import os

extra = os.environ.get("NBCONVERT_EXTRA_TEMPLATE_BASEDIR")
if extra:
    # you can even use `.append()` or `.extend()` on `c.Exporter.extra_template_basedirs`
    c.Exporter.extra_template_basedirs = [
        os.environ.get("NBCONVERT_EXTRA_TEMPLATE_BASEDIR")
    ]

Or maybe the software you use explicitly as nbconvert to not read its config, and hence the need for a patch.

This does not means we cannot add the option regardless, but using the default config capabilities should allow you to modify any config option instead of creating and add-hoc version every time.

If there is a global reason to use ENV variable instead of doing on a per-use-case, we could imaging having a mapping from env variable to configuration option, say NBCONVERT__<classname>__<Option_name>=value.

That would make sens I think, but you also should be able to achieve it now with:

$ cat ~/.jupyter/jupyter_nbconvert_config.py
import os

for k, v in os.environ.items():
    if k.startswith("NBCONVERT"):
        # use __ instead of . as separator in env variable.
        # Warning, case sensitive !
        _, klass, attribute = k.split("__")
        # c.Klass.attribute=v
        setattr(getattr(c, klass), attribute, v)

Allows me to set $ export NBCONVERT__Exporter__extra_template_basedirs='mypath', and this will have the expected effect.

ndmlny-qs

ndmlny-qs commented on Aug 10, 2023

@ndmlny-qs

Thanks for the thoughtful considerations @Carreau. @govinda18 can chime in on their use case, but I can think of one where environment variables would be useful.

If you have an automated deploy scenario for clients, and want all of them to use a centralized extra nbconvert templates directory that you have custom artifacts in, then I see two options available:

  1. in your deploy scripts, you generate the jupyter_nbconvert_config.py file for each client that gets spun up;
  2. have an env variable that nbconvert recognizes on each client.

I suspect that (since it is easy to add/remove environment variables for newly create clients) having nbconvert read environment variables compared to updating client preferences in a python file is the easiest change from a devops perspective. This is of course is just a guess.

Carreau

Carreau commented on Aug 10, 2023

@Carreau
Member

Yes, this is my guess as well, but if that's the case then likely the need is likely not limited to nbconvert, and it might be worth it to patch traitlets to load from env by default in the same way that we have a config file loader and and cli loader ?

Carreau

Carreau commented on Aug 28, 2023

@Carreau
Member

@govinda18 I wanted to know if you had a chance to look at my suggestions and clarify your use case.

Carreau

Carreau commented on Nov 28, 2023

@Carreau
Member

#2075 might help

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      Participants

      @Carreau@govinda18@ndmlny-qs

      Issue actions

        Env variable for extra_template_basedirs · Issue #2026 · jupyter/nbconvert