@jupyterlab/builder builds differently when @jupyterlab dependencies are installed under an alias

[naterush]

Description

I'm an extension author whose finally starting the 4.0 upgrade process. Notably, we'd like to be able to support multiple versions of JupyterLab from within a single version of one Python package -- it makes our lives so much easier, and makes sure we can support all our open source users and paying customers effectively/efficiently!

To do so, we need two different versions of the dependency @jupyterlab/application installed in our package. Thus, we need to install them under an alias -- which results in a package.json that looks something like:

{

"dependencies": {
    ....
    "jupyterlab-application-3": "npm:@jupyterlab/application@^3.5.0",
    "jupyterlab-application-4": "npm:@jupyterlab/application@^4.0.8",
    ....
  },
}

However, when we make this change (and just this change alone) -- our JupyterLab extension no longer works correctly. From looking at the built assets, it looks like a vendors-node_modules_jupyterlab-apputils-4_lib_index_js.195f33f81deda17059ad.js file is now generated in the the

From digging into @jupyterlab/builder, it seems like detecting which packages are bundled vs. not is controlled by sharedPackages definitions.

But I'm wondering if aliases work with @jupyterlab/builder at all -- at runtime, how would it know to resolve to shared packages? Any guidance here would be helpful.

Reproduce

1. This commit works.
2. This commit does not work.

If you pull each of these, and run:

bash dev/setup-jupyterlab-4.0.sh

Then, in a new terminal, start jupyter lab.

In JuypterLab:

1. Make a notebook
2. Look at the toolbar to see that the New Mitosheet button is created vs. not.

On the first commit, Mito works properly. In the second commit, where we move to using aliases, Mito does not work.

Expected behavior

Most specifically: I'd like aliases to work with packages like @jupyterlab/application.

More generally:I'd expect that there is some way of getting two versions of @jupyterlab/application working with rebuild extensions. Is this possible?

[krassowski]

While not answering your question on whether the approach you are attempting could be supported, I wanted to link to a different approach discussed by @bollwyvl in https://discourse.jupyter.org/t/best-way-to-develop-extension-for-jupyterlab-3/21168 which should work already.

[naterush]

@krassowski sweet, I'll check this out in the coming weeks and report back.

Let me know if more context would be helpful for understanding why multi-jupyterlab-version support within one package is so important to us. In general, it reduces our deployment and maintenance costs -- and I think this is probably the case for lots of other extension authors (and certainly users!) as well.

To be clear, makes sense that y'all have to change/upgrade APIs. It would just be really nice to support multiple JuypterLab APIs within one Python package.

[JasonWeill]

@naterush Thank you for opening this! My team, which maintains Jupyter AI and Jupyter Scheduler, tried to create extensions to work with both JupyterLab 3 and 4. In the end, rather than the manual type checking (effectively ignoring types in TypeScript) in the Discourse thread, we created two separate branches. We now have AI and Scheduler 1.x that work with JupyterLab 3, and the main branch of each package (released as 2.x) works with JupyterLab 4. This is more complicated to build and deploy, but we've found that the type safety is more consistent this way.

[JasonWeill]

Accepting this as an enhancement; this is a good thing to have for extension authors who want to build for multiple major Lab versions. This is also likely to be a complex project to build, requiring coordination with multiple developers. If you'd like to discuss this further, we have a weekly call (see our community calendar) that you're welcome to join.

[krassowski]

To be clear, makes sense that y'all have to change/upgrade APIs. It would just be really nice to support multiple JuypterLab APIs within one Python package.

You could have two separate branches which produce artefacts into a single Python package and then decide at install time which one to dump in the appropriate location (which may be a bit more complicated). There is no utility function for this yet but maybe we could have a utility function in a jupyter-packaging/hatch extension or other to help with this.