[JEP 0028] Changes to Configuration System

[Zsailer]

From JEP 0028 (Jupyter Server):

Configuration System

Splitting the server-specific pieces from the classic notebook affects Jupyter's configuration system. This is a non-trivial problem. Changing Jupyter's configuration system affects everyone. We need to consider how to make this transition as painless as possible. At the end of this section, we list some steps that make reduce the friction.

Here is a list of things the changes on the configuration system:

• Move server-specific configuration from jupyter_notebook_config.py|json into jupyter_server_config.py|json.
• Notebook configuration will stay in jupyter_notebook_config.py|json.
• Server extensions configurations move from jupyter_notebok_config.d to jupyter_server_config.d.
• The tornado server and webapp configurable applications move to jupyter_server. They become ServerApplication and ServerWebApp
• The NotebookApp becomes a server extension. It would only load notebook specific configuration/traitlets, from jupyter_notebook_config.py|json.
• Server extensions are found using the jpserver_extensions trait instead of the nbserver_extensions trait in the ServerApp.
• Extension configuration files in jupyter_server_config.d must be enabled using the jpserver_extensions trait. They are enabled by JSON config files in jupyter_server_config.d.
• Extensions can create their own configuration files in {sys-prefix}/etc/jupyter/ or ~/.jupyter, i.e. jupyter_<my-extension>_config.py|json.
• General jupyter_config.py|json files must update to set server-specific configuration using the ServerApp and notebook specific configuration using NotebookApp.

Some traits will stay in jupyter_notebook_config.py|json. Here is a list of those traits (everything else moves to server config files):

• extra_nbextensions_path
• enable_mathjax
• max_body_size
• max_buffer_size
• notebook_dir
• mathjax_url
• get_secure_cookie_kwargs
• mathjax_config
• ignore_minified_js

Here are some steps we can take to reduce the friction for transitioning:

• Copy (not move) jupyter_notebook_config.py|json to jupyter_server_config.py|json
• Copy (not move) jupyter_notebook_config.d/ to jupyter_server_config.d.
• NotebookApp becomes ServerApp in all copied files.
• Leftover server traits in jupyter_notebook_config.py|json get ignored when the notebook extension is initialized. Server traits are only read from jupyter_server_config.py|json.
• Document like crazy!

For an thorough overview of the configuration system and a side-by-side diff between the (current) notebook and (future) server config layout, take a look at Zsailer/jupyter_config_overview.

Opening for further discussion.

[Zsailer]

Previous comments from JEP #28

@mpacer : I think what we should do is make a change to JupyterApp from jupyter_core that allows us to declare additional configuration file names that should be picked up. Currently you can only address one app at time (or all apps with jupyter_config.py) based on the value of the app's class attribute name.

Additionally, we could make a Configurable class (NotebookAppConfig) that inherits from the NotebookApp and overwrites all of it's webserver capabilities so that we can continue to get access to the existing NotebookApp.nbserver_extension information. But again… that will be meaningless if we can't read in people's jupyter_notebook_config.py, jupyter_notebook_config.json, and jupyter_notebook_config.d/extension.json.

@rgbkrk : Adding on -- with Netflix's current deployments we deploy classic, jupyterlab, and nteract -- we want all of them and in many cases want to share config (so that ContentsManagers are the same and additional server extensions are available for APIs, like with bookstore).

Even if I was elsewhere, I'd still want to have a consistent deterministic config setup even with multiple jupyter apps attached to the server.

@Zsailer: @mpacer this is a good idea--add a trait to the JupyterApp for listing configuration files that should get loaded (and in what order of precedence). That would certainly help with avoiding breaking everyone's configuration system in this transition.

I missed jupyter/notebook#4376 when trying to research all the threads around configuration. Sorry about that! I'll need to think about this a bit more for a new draft. (I'm working on adding flow diagram/visualizations showing the proposed configuration models to this proposal to help others follow the conversation as well).

That said, I know that jupyterlab has moved towards a configuration model where each extension has its own config file a conf.d. I was modeling the jupyter server (and its extensions) after this model. But, this conversation has made me realize this needs a lot more thought. Specifically, I need to think a lot more about Netflix's use-case here... thanks for sharing @rgbkrk

[Zsailer]

Closing, due to inactivity.