Built-in extensions using federated dependencies

[jasongrout]

References

This fixes the last half of #9289. It allows built-in extensions to use dependencies shared by federated extensions. It also cleans up the built-in extension configuration.

Code changes

• Load federated extensions in a bootstrap phase, before the main application imports compiled plugins and starts the application. This ensures that if a compiled plugin requires something provided by a federated extension, it will be able to get it.
• Treat compiled extensions (including core extensions) and federated extensions similarly when it comes to sharing config. We always default to sharing all direct dependencies, and we allow for any plugin to have a customized jupyterlab.sharedConfig key listing sharing config that overrides these defaults.

We are careful to merge the extension sharing config for the compiled build. We never override the core set of resolutions (those are always shared), and any shared config that says to bundle a dependency takes precedence over a config saying not to bundle a dependency.

User-facing changes

Compiled extensions can use federated dependencies. For example, with this, a user should be able to jupyter labextension install an ipywidgets plugin with the jupyterlab_widgets federated extension base widget package. The ipywidgets plugin should be able to use the ipywidgets manager provided by the federated extension.

Backwards-incompatible changes

[jupyterlab-dev-mode]

Thanks for making a pull request to JupyterLab!

To try out this branch on binder, follow this link:

[jasongrout]

A few tests are failing because we have a new file that will appear in staging when we make a release, but since it's not there now, the python commands to copy it over to share/jupyter/lab/staging are failing.

To test this:

conda create -y -n jlab-9289 python nodejs
conda activate jlab-9289

pip install --pre jupyterlab==3.0.0rc7

cd $(mktemp -d)
git clone https://github.com/jasongrout/provider.git
git clone https://github.com/jasongrout/consumer.git

pushd provider
# makes and installs a federated extension
pip install .
# build so we can install in consumer
jlpm install
jlpm run build
popd

pushd consumer
jlpm add `realpath ../provider` # or manually edit package.json to give the absolute path to provider
jupyter labextension install .
popd

Check things with jupyter labextension list.

Now do jupyter lab and notice that there are errors in the console:

Plugin 'consumer' failed to activate.
Error: No provider for: provider:plugin.

and not the expected logs of JupyterLab extension provider is activated!, JupyterLab extension consumer is activated!, Provider token is PROVIDER_STRING.

Now we apply this PR manually. For the patches below, skip any patches that don't apply.

# patch command.py
pushd [site-packages]/jupyterlab
curl https://patch-diff.githubusercontent.com/raw/jupyterlab/jupyterlab/pull/9310.patch | patch -p2

# patch index.js, bootstrap.js, and webpack.config.js
cd [site-packages]/jupyterlab/staging
curl https://patch-diff.githubusercontent.com/raw/jupyterlab/jupyterlab/pull/9310.patch | patch -p2

popd

jupyter lab clean
jupyter lab build

Now run jupyter lab. No errors, and we get the message from the consume plugin indicating it correctly retrieved the object provided by the provider plugin:

Provider token is PROVIDER_STRING

[jtpio]

Thanks Jason!

To test this:

Following these steps locally lead to the same observations 👍

I also tested this PR using the same setup as the one described in #9300 (comment), with an extra token package that exports the token between the consumer and the provider.

In this case, the dependencies of the provider (namely the token package) are correctly added to the shared scope, and the consumer (installed as federated) is able to specify the token package (a dependency of both the consumer and the provider) as sharedPackages to enable sharing.

---

using the same setup as the one described in #9300 (comment)

For extra context and for those interested, here is the setup with a concrete example:

• jupyterlab-topbar defines the token
• jupyterlab-topbar-extension provides the token, and is installed as an extension built into JupyterLab (jupyter labextension install)
• jupyterlab-system-monitor consumes the token and is installed as federated. It is able to specify the package that exports the token as sharedPackages as follows: https://github.com/jtpio/jupyterlab-system-monitor/blob/2620b915d28b800d4eeade1020e8cb0b24114e83/packages/system-monitor/package.json#L72-L77

With this PR, and in particular d17aca7 and 721c70d, dependencies of the extension are also added to the shared scope. Which makes it possible for the consumer package installed as federated to define its sharedPackages.

[jasongrout]

@jtpio - is that a positive review? If it is, I'll merge.

(or even better, you can merge)

[jtpio]

Thanks!