Widgetslabextension package #2806

martinRenou · 2020-03-04T15:12:55Z

Sorry in advance if an issue was already opened for this.

After a long discussion with @maartenbreddels yesterday we agreed that installing an interactive widgets library for JupyterLab is too complicated for users, and very error-prone. We think it might scare people off and result in giving up trying to use our extensions.

One of the issues we discussed was that when doing:

conda install ipyleaflet

jupyter labextension install jupyter-leaflet

You might get a Python package that is incompatible with the JS package, resulting in the Widget model not found error in JupyterLab.

@maartenbreddels @mariobuikhuizen and I think @vidartf went around this specific issue in their extensions by putting a tgz of the JS package under share/jupyter/lab/extensions, allowing JupyterLab to find the package and rebuild itself (either from the GUI or doing jupyter lab build).

This ensures having a version of the JS package compatible with the Python package. One would argue it makes the Python package dependent on the JS package, and I would agree it's not a good thing, but usability would be improved.

I would like to propose doing the same thing in ipywidgets, with a new widgetslabextension package (widgetsnbextension counterpart) which installs the tgz at the right place.
The only thing I am not sure about is how this package would be installed. Should it be a JupyterLab dependency? An ipywidgets dependency? Should the user install it manually?

I can come up with a PR myself if nobody is against it.

The text was updated successfully, but these errors were encountered:

jasongrout · 2020-03-04T15:32:48Z

I've been going back and forth on this thought too over the last year or two. This also follows on other discussions we had at the widgets dev sprint about splitting up how our dependencies work, and in particular how the widgetsnbextension package works and what pulls it in.

Another factor here is that JupyterLab is improving its extension manager support, so it should be much easier to find and install an extension from within JupyterLab. Arguably that is the more theoretically correct solution, since the frontend and backend really are separate places to install things.

On the other hand, having another way to distribute the js is probably not a bad thing, other than being confusing that there is one more way to install the lab extension (do I go through the lab extension manager, or through the lab command line, or this new python package?)

martinRenou · 2020-03-04T15:58:56Z

From a user perspective, being able to have a working notebook just doing:

pip install jupyter ipywidgets
# Or 
conda install jupyter ipywidgets

is super nice. I would love to see the same happen with JupyterLab.

I know it is more complicated than that with JupyterLab because of the deduplication, but I guess we can at least allow to turn the following:

conda install jupyterlab ipywidgets ipyleaflet ipyvolume ipycanvas

jupyter labextension install @jupyter-widgets/jupyterlab-manager jupyter-leaflet ipyvolume ipycanvas

into a simpler:

conda install jupyterlab ipywidgets ipyleaflet ipyvolume ipycanvas

jupyter lab build

Not only it is simpler, but it is also safer because there is no mismatch between the Python and JS packages version.

This also follows on other discussions we had at the widgets dev sprint about splitting up how our dependencies work, and in particular how the widgetsnbextension package works and what pulls it in.

I guess the same kinds of discussions would rise if we make a widgetslabextension. I cannot find what pulls widgetsnbextension, would you mind reminding it to me?

I guess I'm in favor of usability, and have widgetslabextension pulled automatically. The same way it is done right now for widgetsnbextension.

jasongrout · 2020-03-04T16:08:55Z

Right now ipywidgets depends on widgetsnbextension. The thought from @SylvainCorlay was that we break that dependency, I think? Maybe it was that we have a ipywidgets base package that just installs the kernel python package, then an ipywidgets package that installs ipywidgets base, widgetsnbextension, and a lab extension python package.

jasongrout · 2020-03-04T16:10:24Z

into a simpler:

Note that this is only simpler if JupyterLab and your kernel are in the same python environment. If you are running multiple kernels from different environments, things are more complicated again.

Though probably the most common case by far is jlab and the kernel being in the same environment.

martinRenou · 2020-03-04T16:12:34Z

Right now ipywidgets depends on widgetsnbextension. The thought from @SylvainCorlay was that we break that dependency, I think? Maybe it was that we have a ipywidgets base package that just installs the kernel python package, then an ipywidgets package that installs ipywidgets base, widgetsnbextension, and a lab extension python package.

I like the idea.

An important question then is: does conda install ipywidgets install the Python-only package or the nb/labextension as well?

SylvainCorlay · 2020-03-04T16:17:05Z

I would like ipywidgets to not depend on anything that relates to the server environment.

Maybe the only other package should be called jupyter_widgets and include (all) the front-end assets. At the moment, these assets differ for JLab and Notebook, but could be both included in jupyter_widgets. Hopefully, we will end up with a more uniform format for all the front-ends including nteract, Voilà, jupyterlab etc.

Also, jupyter_widgets (or widgetsnbextension at the moment) should not have a dependency on the notebook server or anything. Ideally it should only include the JS assets as data files.

martinRenou · 2020-03-04T16:20:16Z

Concerning the name, doesn't it bring more confusion? I think of libraries like ipyleaflet for which jupyter-leaflet refers to the JS only package?

SylvainCorlay · 2020-03-04T16:23:31Z

This may actually align the planets, since the jupyter_leaflet package splot off ipyleaflet would basically include the jupyter-leaflet JS assets packaged for the different jupyter frontends.

jasongrout · 2020-03-04T16:23:49Z

Perhaps ipywidgets_frontend to make it even more clear?

SylvainCorlay · 2020-03-04T16:24:23Z

ipywidgets_frontend

it is also the xwidgets frontend 😄

jasongrout · 2020-03-04T16:24:38Z

Actually, I do like jupyter_widgets, in that the "ipy" is specific to the python kernel, but the jupyter_widgets package is only for frontends, and is agnostic about the kernel language, etc.

SylvainCorlay · 2020-03-04T16:26:04Z

One important aspect of jupyter_widgets is that it should not depend on notebook or jupyterlab. It is only static assets, to be picked up by frontends.

SylvainCorlay · 2020-03-04T16:26:56Z

or jupyter_widgets_frontend...

jasongrout · 2020-03-04T16:27:03Z

So deprecate widgetsnbextension, and replace it with jupyter_widgets which contains the nbextension and lab extension as data files?

I still think it would be nice to have an ipywidgets_base with just the python, and ipywidgets pulls in both the kernel and the frontend packages. That's definitely more convenient.

SylvainCorlay · 2020-03-04T16:28:17Z

I still think it would be nice to have an ipywidgets_base with just the python, and ipywidgets pulls in both the kernel and the frontend packages. That's definitely more convenient.

meh. I think that people can understand that they need to install the "frontend" (actualy more than having to install the "notebook extension").

Or we keep the dependency to the front-end, (which can be installed without harm in the kernel environment) as long as the front-end package is only static assets and does not depend on jupyer_server, notebook, or lab...

jasongrout · 2020-03-04T16:31:02Z

This may actually align the planets

I see what you did there... :)

SylvainCorlay · 2020-03-04T16:31:39Z

This may actually align the planets

I see what you did there... :)

The moons?

martinRenou · 2020-03-04T16:33:55Z

Or we keep the dependency to the front-end, (which can be installed without harm in the kernel environment) as long as the front-end package is only static assets and does not depend on jupyer_server, notebook, or lab...

👍

maartenbreddels · 2020-03-04T19:44:38Z

I still think it would be nice to have an ipywidgets_base with just the python, and ipywidgets pulls in both the kernel and the frontend packages.

I agree, or, what people use a lot is a core postfix, ipywidgets-core.

So ipywidgets would depend on ipywidgets-core and jupyter-widgets (and possibly notebook and lab?), and jupyter-widgets would only contain all js bundles (lets not split that up).

Basically ipywidgets is there to get you everything, including the kitchensink. But for those that care about setting up clean environments, they would install ipywidgets-core in the kernels, and jupyter-widgets in the jupyter-server side?

jasongrout · 2020-03-04T19:48:28Z

(and possibly notebook and lab?)

No. If they only want notebook, we shouldn't be pulling in lab, and vice versa.

Other than that, it sounds okay to me, though it would be nice if we had a better name than jupyter_widgets to convey that it is the package for frontends.

SylvainCorlay · 2020-03-04T19:50:56Z

I really don't like -core and -base suffixes which don't say what things are.

Keeping the ipywidgets -> jupyter_widgets_frontend dependency is find to me as long as the latter only puts things in PREFIX/share and PREFIX/etc and does not have any dependency.

maartenbreddels · 2020-03-04T20:00:39Z

I really don't like -core and -base suffixes which don't say what things are.

Actually, I don't care so much how we name and, and almost nobody will care I think, or pay attention to it. It's more an internal thing to split things up right?

But I'm also ok not yet splitting off a yet. I think the most important step is to get the tgz bundle in so jupyter lab picks it up.

jasongrout · 2020-03-04T20:16:15Z

Now that widgets supports JLab 2, we can renew our push towards ipywidgets 8, and make this one of the features?

SylvainCorlay · 2020-03-04T21:04:26Z

Actually, I don't care so much how we name and, and almost nobody will care I think, or pay attention to it. It's more an internal thing to split things up right?

For matplotlib-base, I found this very confusing. I don't know what's not in it etc...

maartenbreddels · 2020-03-04T21:09:26Z

But as a user, I'm not so aware of it. I've been it pass by my screen when installing with a pip install, but I have no idea what it does, nor do I care or have to care right?

SylvainCorlay · 2020-03-04T21:22:14Z

But as a user, I'm not so aware of it. I've been it pass by my screen when installing with a pip install, but I have no idea what it does, nor do I care or have to care right?

Well, you tend to be super quickly because of all the size of the pyqt dependency, or because the requirement of pyqt conflicts with your own...

martinRenou · 2020-03-05T07:31:44Z

Now that widgets supports JLab 2, we can renew our push towards ipywidgets 8, and make this one of the features?

I could do it. Maybe we could split it in two stages?

1:

deprecate widgetsnbextension
create jupyter_widgets_frontend which installs the nbextension and the labextension
make ipywidgets dependent on jupyter_widgets_frontend

2:

create ipywidgets-core (or another name)
make ipywidgets dependent on ipywidgets-core

martinRenou · 2020-03-05T07:33:32Z

I think I can come up with a PR for the first stage today

wolfv · 2020-03-05T10:53:51Z

One annoying issue that will remain is that having multiple version of the frontend extension is still not easy. (e.g. some older bqplot version for python 2). so maybe the frontend version could be more strictly versioned right in the name (e.g. bqplot-frontend[5.2])

jasongrout · 2020-03-09T21:36:57Z

I was looking at this more today. jupyter_widgets_frontend sort of sounds like we are installing a new frontend, like a new notebook or something. What about jupyter_widgets_extension? That's not particularly clear either.

jasongrout · 2020-03-09T21:39:17Z

Or I'm getting to like better just jupyter_widgets, with the one-line description saying it is the frontend extensions needed to run jupyter widgets.

jasongrout · 2020-03-09T22:01:21Z

Another thought on this: right now, JupyterLab is getting more intelligent with how to install compatible extensions. For example, we just had a JupyterLab 2.0 release. This did not require any ipywidgets python package releases. Instead, jupyter labextension install @jupyter-widgets/jupyterlab-manager automatically finds the right version of the npm package and installs it, so the same instructions for installing the lab extension still automatically work for both jlab 1 and 2.

How would this JupyterLab 2.0 release go if we had this jupyter_widgets package that bundled the js? Let's assume we had jupyter_widgets 1.0 that supports jlab 1. Now we release jlab 2 and people upgrade to it. Now people either need to install jupyter_widgets 2.0, or manually install the jupyter lab extension to override the js bundled in jupyter_widgets 1.0. Also, how would people with JLab 1.0 know to install jupyter_widgets 1.0 instead of the latest jupyter_widgets 2.0? And if ipywidgets depended on jupyter_widgets, that now means that ipywidgets has to bump its version number too, and which should it depend on, 1.0 or 2.0 or both?

vidartf · 2020-03-10T13:43:34Z

How would this JupyterLab 2.0 release go if we had this jupyter_widgets package that bundled the js? Let's assume we had jupyter_widgets 1.0 that supports jlab 1. Now we release jlab 2 and people upgrade to it. Now people either need to install jupyter_widgets 2.0, or manually install the jupyter lab extension to override the js bundled in jupyter_widgets 1.0.

The manual install = jupyter labextension install @jupyter-widgets/jupyterlab-manager right? In which case it will be the same instruction as for the current system?

Also, how would people with JLab 1.0 know to install jupyter_widgets 1.0 instead of the latest jupyter_widgets 2.0?

If I create such an env (env with lab 2.0, npm pack @jupyter-widgets/jupyterlab-manager@2.0.0 in the share dir), lab gives me a warning on start-up:

[W 13:28:48.010 LabApp]
    "@jupyter-widgets/jupyterlab-manager@2.0.0" is not compatible with the current JupyterLab
    Conflicting Dependencies:
    JupyterLab                        Extension      Package
    >=1.2.1 <1.3.0                    >=2.0.0 <3.0.0 @jupyterlab/application
    >=1.2.2 <1.3.0                    >=2.0.0 <3.0.0 @jupyterlab/notebook
    >=1.2.1 <1.3.0                    >=2.0.0 <3.0.0 @jupyterlab/rendermime
    >=1.5.0 <1.6.0                    >=2.0.0 <3.0.0 @jupyterlab/rendermime-interfaces
    >=4.2.0 <4.3.0                    >=5.0.0 <6.0.0 @jupyterlab/services

Running jupyter labextension update --all will then downgrade the manager:

>jupyter labextension update --all
Updating @jupyter-widgets/jupyterlab-manager to version 1.1.0

I'm not sure if all of this will be more or less intuitive than it is today, but I'm trying to say that we can at least build a set of instructions that will work.

And if ipywidgets depended on jupyter_widgets, that now means that ipywidgets has to bump its version number too, and which should it depend on, 1.0 or 2.0 or both?

This is trickier. I'm not entirely sure how this version linking is proposed to be done, but if we are version linking them (ipywidgets@X --> jupyter-widgets@Y.*), then that version link should ideally represent the protocol version? Hm...

jasongrout · 2020-03-10T18:16:28Z

One mismatch here is that the right frontend things to be installed depends on the frontend, not on the kernel-side ipywidgets. So really it is most natural to have these two installations to be independent of each other.

jasongrout · 2020-03-10T18:27:02Z

So what if the new instructions were to do conda install ipywidgets jupyter_widgets? And jupyter_widgets would constrain the version of JupyterLab and notebook to be compatible.

jasongrout · 2020-03-20T14:02:31Z

And jupyter_widgets would constrain the version of JupyterLab and notebook to be compatible.

How could we constrain the version in pip? We'd need to say somehow that if you have notebook installed, it should be in this version range, but not to explicitly install the notebook if you don't already have it installed.

jasongrout · 2020-03-20T14:05:54Z

pip does have a constraints file concept: https://pip.readthedocs.io/en/stable/user_guide/#constraints-files - but that doesn't apply for installation requirements, I think.

jasongrout · 2020-03-20T14:22:20Z

Two thoughts:

pip will be releasing (in 2020) a version that has an actual dependency solver. I don't know if this will include something like optional constraints on the environment. If it does, then that brings it up to pare with conda for our usecase.
We could possibly use the setup.py extra requirements for this case. Perhaps pip install jupyter_widgets[notebook] would depend on the right notebook versions, and pip install jupyter_widgets[jupyterlab] would depend on the appropriate jupyterlab version. In this case, I think pip install jupyter_widgets would still install everything just the same (it's just copying files to a directory), but the install wouldn't be checking the lab or notebook versions and upgrading/downgrading if necessary.

ianhi · 2020-07-13T22:18:16Z

I don't think I fully grasp the nuance of the dependency resolution problems but given:

One mismatch here is that the right frontend things to be installed depends on the frontend, not on the kernel-side ipywidgets

is there any possibility of jupyterlab-manager being included in jupyterlab itself? This seems like it would automatically would keep the frontend versions in sync.

Big 👍 to anything the removes the need to run jupyter labextension install @jupyter-widgets/jupyterlab-manager as this can be very confusing. For example here are some quotes from a colleague I was helping to get interactive plot working today:

why don't i get an error message then?

your mystifying incantation works

why does jupyter notebook support this out of the box? and not jupyterlab

i might need to be the Luddite who sticks with notebooks

Related to the first quote is there any way to raise an error in python when the widget versions don't match or jupyterlab-manager is not present? The failure state of mismatched javascript version or equivalent can be very opaque especially if someone doesn't know about the webdev console.

jasongrout · 2021-02-18T19:17:57Z

The status as of now (ipywidgets 7.6) is that ipywidgets requires widgetsnbextension (which in turn requires notebook), and also requires jupyterlab_widgets (which does not require jupyterlab).

For ipywidgets 8, the plan is to have ipywidgets require widgetsnbextension (which would not require notebook) and jupyterlab_widgets (which again would not require jupyterlab).

The end result is that pip install ipywidgets with ipywidgets 7.6 makes it work in a local notebook and JupyterLab 3, with no extra steps. For backwards compatibility, this will also pull in notebook, but won't in ipywidgets 8.

As such, I think the main concerns raised here are now resolved. If anyone feels otherwise, please reopen or comment below.

jasongrout added this to the Future milestone Mar 4, 2020

jasongrout added the discussion label Mar 4, 2020

martinRenou mentioned this issue Mar 5, 2020

Add jupyter_widgets #2807

Closed

martinRenou mentioned this issue May 26, 2020

JupyterLab extension files not bundled into pip package jupyter-widgets/widget-cookiecutter#58

Open

ceball mentioned this issue May 27, 2020

Automatically install lab extension when installing the python package? bokeh/jupyter_bokeh#105

Closed

jasongrout closed this as completed Feb 18, 2021

sspathak mentioned this issue Jul 12, 2024

Fixed bug caused due to outdated versions jhamman/streaming-zarr#2

Open

Widgetslabextension package #2806

Widgetslabextension package #2806

Comments

martinRenou commented Mar 4, 2020

jasongrout commented Mar 4, 2020

martinRenou commented Mar 4, 2020

jasongrout commented Mar 4, 2020

jasongrout commented Mar 4, 2020

martinRenou commented Mar 4, 2020

SylvainCorlay commented Mar 4, 2020

martinRenou commented Mar 4, 2020

SylvainCorlay commented Mar 4, 2020

jasongrout commented Mar 4, 2020

SylvainCorlay commented Mar 4, 2020

jasongrout commented Mar 4, 2020

SylvainCorlay commented Mar 4, 2020

SylvainCorlay commented Mar 4, 2020

jasongrout commented Mar 4, 2020

SylvainCorlay commented Mar 4, 2020 • edited Loading

jasongrout commented Mar 4, 2020

SylvainCorlay commented Mar 4, 2020

martinRenou commented Mar 4, 2020

maartenbreddels commented Mar 4, 2020

jasongrout commented Mar 4, 2020

SylvainCorlay commented Mar 4, 2020 • edited Loading

maartenbreddels commented Mar 4, 2020

jasongrout commented Mar 4, 2020

SylvainCorlay commented Mar 4, 2020

maartenbreddels commented Mar 4, 2020

SylvainCorlay commented Mar 4, 2020

martinRenou commented Mar 5, 2020

martinRenou commented Mar 5, 2020

wolfv commented Mar 5, 2020

jasongrout commented Mar 9, 2020

jasongrout commented Mar 9, 2020

jasongrout commented Mar 9, 2020

vidartf commented Mar 10, 2020 • edited Loading

jasongrout commented Mar 10, 2020

jasongrout commented Mar 10, 2020 • edited Loading

jasongrout commented Mar 20, 2020

jasongrout commented Mar 20, 2020

jasongrout commented Mar 20, 2020 • edited Loading

ianhi commented Jul 13, 2020

jasongrout commented Feb 18, 2021 • edited Loading

SylvainCorlay commented Mar 4, 2020 •

edited

Loading

SylvainCorlay commented Mar 4, 2020 •

edited

Loading

vidartf commented Mar 10, 2020 •

edited

Loading

jasongrout commented Mar 10, 2020 •

edited

Loading

jasongrout commented Mar 20, 2020 •

edited

Loading

jasongrout commented Feb 18, 2021 •

edited

Loading