Update and reinstate Extension Manager

[blink1073]

In #8802 we removed the extension manager and associated documentation, since we'll need to update it to target pip and conda. This task tracks re-instating the manager and docs.

[goanpeca]

I would like to work on this @blink1073 :-)

[blink1073]

I would like to work on this @blink1073 :-)

Sounds good, we can talk after i18n work is merged

[goanpeca]

@isabela-pf we could use this issue to include the UI/UX discussion as well (maybe?) ?

[goanpeca]

Sounds good, we can talk after i18n work is merged

Let's talk @blink1073

[goanpeca]

For reference:

jupyter labextension install @jupyter-widgets/jupyterlab-manager@3.0.0-alpha.1

[stonebig]

I don't catch:

• do "pip install jupyterlab" people need to do also "jupyter labextension install @jupyter-widgets/jupyterlab-manager@3.0.0-alpha.1" ?
• is "jupyter labextension install" still needed on user side ?

[jasongrout]

I think we need to have a clear story about this in 3.0. Right now, the extension manager is active.

[jasongrout]

Some thoughts about what we can do for 3.0:

1. Make the listing of installed packages in the extension manager also list federated extensions
2. Add some (collapsible?) text in the extension manager explaining that the extensions managed from the extension manager do not include federated extensions, with a link to a docs page clearly explaining federated extensions vs classic extensions.

[echarles]

Hi @jasongrout Working on this. I will post a screenshot of the changes hopefully tomorrow.

Make the listing of installed packages in the extension manager also list federated extensions

Looks like the federated extensions can be pulled from

jupyterlab/dev_mode/index.js

Line 81 in 4d898e0

PageConfig.getOption('federated_extensions')

I need to check if the non-federated ones are discovered like before.

Maybe they should be displayed in separated collapsible sections?

Also merging #9203 will help me with my local env for test.

Add some (collapsible?) text in the extension manager explaining that the extensions managed from the extension manager do not include federated extensions, with a link to a docs page clearly explaining federated extensions vs classic extensions.

Does that doc content already exist? I can see https://jupyterlab.readthedocs.io/en/latest/developer/extension_dev.html?#goals-of-the-federated-extension-system

[jasongrout]

Hi @jasongrout Working on this. I will post a screenshot of the changes hopefully tomorrow.

Awesome! I started looking at what was needed as well, just to double-check our information.

Looks like the federated extensions can be pulled from

Oh, nice. I realized that the list of extensions returned from the server api did not include federated extensions, and it wasn't immediately clear what needed to change to have the api updated.

Also merging #9203 will help me with my local env for test.

We just merged it. I'll release a new RC with it as well.

Does that doc content already exist?

No. I can work on that. I think there should be a separate user page talking about the extension system (as opposed to a developer page).

[jasongrout]

Does that doc content already exist?

No. I can work on that. I think there should be a separate user page talking about the extension system (as opposed to a developer page).

Actually, it looks like we have a user extensions page: https://jupyterlab.readthedocs.io/en/stable/user/extensions.html - let's refer to that and I'll update the text.

[oeway]

Hi, I would like to add that it would be nice to distinguish plugins installed from npm/pypi and plugins installed locally for development in the Extension Manager, perhaps disable the install options for locally installed plugins.

Currently, if I install an extension from a local folder, then go to the extension manager click Uninstall then click install again, the extension will enter a bad state. If I try to install it again from my local folder, it keep saying
RuntimeError: JupyterLab failed to build and if I dig into the log I found Error: Cannot find module 'my_extension/package.json. I had to create a new conda environment and later @telamonian told me on gitter that I can use jupyter lab clean --all to solve it.

[jasongrout]

I'm working on updating the extension documentation at #9221

[jasongrout]

@echarles - since you are already touching the extension manager code, can you take care of #9170 if it's easy?

[fcollonval]

@echarles I push #9222 to address #9170.

[echarles]

Thx @fcollonval for #9222

[echarles]

@jasongrout

I have looked closer at the options to install/link/build the static and dynamic extensions. I have also looked at the code sections that are invoked when you type on the CLI jupyter labextension list or when the browser requests the endpoint /lab/api/extensions. I was feeling a bit more confident to start with the implementation of this issue.

The first steps I had were something like:

• Add a collapsible section that would show text and a link to the doc to explain the difference between static and dynamic (another name for static could be built-in - I don't have for now a better name for dynamic).
• Add a collapsible section with label Installed (dynamic) that would be populated from PageConfig.getOption('federated_extensions')`
• Change the collapsible section Installed label to Installed (static)
• Change the collapsible section Discover label to Discover (static)

I have installed an extension via pip which shows well via jupyter labextension list

jupyter labextension list
JupyterLab v3.0.0rc6
...envs/quirkshop-dev/share/jupyter/labextensions
        quirkshop-jlab3-react v0.1.0 enabled OK

Following my routing, I have begin to develop on master with jupyter lab --dev-mode --watch and, prolly expected:

• The extensions are not loaded in the UI.
• Not listed by the /lab/api/extensions endpoint.
• PageConfig.getOption('federated_extensions') object is empty
• The page config in the index.html shows federated_extensions: []

We had a solution to enable extensions in dev_mode for jlab2 with the jupyterlab.externalExtensions (read on https://jupyterlab.readthedocs.io/en/stable/developer/extension_dev.html)

Any idea how to get this on jlab3 - This is pretty needed for me to get this feature as I need to change core source while having extensions enabled to see them in the extension manager.

[jasongrout]

I think externalExtensions still works? At least, it is still in the code:

jupyterlab/buildutils/src/ensure-repo.ts

Line 191 in c9589d6

corePackage.jupyterlab.externalExtensions

jupyterlab/dev_mode/webpack.config.js

Line 28 in 84bcb9a

Object.keys(externalExtensions)

[echarles]

Side note: The current extension manager is still working well for jlab3 compatible extensions published in npm (install/list/...)

[jasongrout]

Thanks for distinguishing between federated extensions and non-federated extensions in the UX. I think it's important that you can only install/uninstall non-federated extensions (actually, I think jlab should keep track of which extensions it has installed from npm, and only allow uninstalling those from the UX. Otherwise you might be just deleting part of a python package, etc., messing up the python packaging).

[echarles]

I think externalExtensions still works? At least, it is still in the code:

Yep, just tried that and we are in the middle ground here. With that externalExtensions in the dev_mode package.json, the extension is loaded in dev mode, but the PageConfig federated_extensions remains empty (both in the index.html and in the PageConfig.getOption('federated_extensions') object).

[echarles]

Thanks for distinguishing between federated extensions and non-federated extensions in the UX. I think it's important that you can only install/uninstall non-federated extensions (actually, I think jlab should keep track of which extensions it has installed from npm, and only allow uninstalling those from the UX. Otherwise you might be just deleting part of a python package, etc., messing up the python packaging).

Just WIP and a bit blocked because of my previous comment...

You talk about federated extensions. Is this what we gonna keep in the doc? If we were to standardize on another terminology, it would be good to align the source code also to keep the same nomenclature.

I push a bit on this as it has taken me time to make sense of the options. I think a clear nomenclature and terminology usage will make life easier for users and developers.

[jasongrout]

You talk about federated extensions. Is this what we gonna keep in the doc? If we were to standardize on another terminology, it would be good to align the source code also to keep the same nomenclature.

I agree. I'm sort of all over the place right now with terminology, trying out different terms to see how they feel, so I apologize for confusion. I still think the most user-friendly documentation probably still doesn't need to use the term "federated"? I'm still writing and rewriting that doc.

[echarles]

Thought a bit more, and I can hard-code in dev mode the value of the federated_extensions key in the PageConfig. I guess we can release an RC to see if it really works.

Any other idea?

[jasongrout]

I think there's no reason we can't have federated extensions in dev mode. (In fact, I thought it was working).