Fix IPython progress bar in notebook versions >= 4 #1088

jgosmann · 2016-06-10T21:07:18Z

Description:
Updates the Jupyter notebook extension to be compatible with notebook and ipywidget versions >= 4, including notebook 4.2 and ipywidgets 5. Also gives a better exception when ipywidgets is not installed.

Motivation and context:
They keep changing their API. 😡
Fixes #1085. Adresses discussion in #1084 (but not the issue itself).

How has this been tested?
Manually by creating virtualenvs with different IPython/Jupyter/ipywidgets versions.

Where should a reviewer start?
The single file that has been changed. duh

How long should this take to review?

Average (neither quick nor lengthy)

Types of changes:

Bug fix (non-breaking change which fixes an issue)

Checklist:

I have read the CONTRIBUTING.rst document.
I have updated the documentation accordingly.
I have included a changelog entry.
I have added tests to cover my changes.
All new and existing tests passed.

jgosmann · 2016-06-10T21:11:19Z

As said in Document nengo.ipynb #1084 there should be some more documentation, but I don't think it needs to be part of this PR.
I don't see a good way to do automated testing of this.
Waiting for the tests to pass on Travis (don't expect them to fail).

tbekolay · 2016-06-22T21:54:13Z

Looked this over and tested it in the newest version of jupyter. Doesn't work in master, works in this branch, so this looks good to me! Will merge on second review... nominating @drasmuss as this is pretty small code-wise and would be good to ensure it works in Windows.

drasmuss · 2016-06-23T16:54:18Z

When I run this I don't get any progress bar, and this error shows up in the console:

[W 12:47:44.292 NotebookApp] 403 GET /static/widgets/js/widget.js?v=20160623124723 (::1): widgets\js\widget.js is not in root static directory
[W 12:47:44.298 NotebookApp] 403 GET /static/widgets/js/widget.js?v=20160623124723 (::1) 6.52ms referer=http://localhost:8888/notebooks/spa/convolution.ipynb

Note: that is if I use the version of the notebook from the repo. If I convert the notebook to 4.0 (by saving it) and then run it, there is no error message but still no progress bar.

jgosmann · 2016-06-23T17:11:27Z

Can you give me your pip freeze output?

drasmuss · 2016-06-23T17:13:31Z

>pip freeze
backports.shutil-get-terminal-size==1.0.0
colorama==0.3.6
cycler==0.10.0
decorator==4.0.10
flake8==2.5.4
ipykernel==4.3.1
ipython==4.2.0
ipython-genutils==0.1.0
ipywidgets==4.1.1
Jinja2==2.8
jsonschema==2.5.1
jupyter==1.0.0
jupyter-client==4.3.0
jupyter-console==4.1.1
jupyter-core==4.1.0
MarkupSafe==0.23
matplotlib==1.5.1
mccabe==0.4.0
mistune==0.7.2
nbconvert==4.2.0
nbformat==4.0.1
-e git+https://github.com/nengo/nengo@6ba0aa91ac1ef5ee1ee7ef470f0e25a4418ab012#egg=nengo
-e git+https://github.com/nengo/nengo_gui@f5958e56919b5d13278059993014e2ec2bebcef1#egg=nengo_gui
notebook==4.2.1
numpy==1.10.4
path.py==0.0.0
pep8==1.7.0
pickleshare==0.7.2
py==1.4.31
pyflakes==1.0.0
Pygments==2.1.3
pyparsing==2.1.0
pyreadline==2.1
pytest==2.8.7
python-dateutil==2.5.0
pytz==2015.7
pyzmq==15.2.0
qtconsole==4.2.1
simplegeneric==0.8.1
six==1.10.0
Sphinx==1.3.1
tornado==4.3
traitlets==4.2.1
wheel==0.29.0

This is python 3.5, anaconda

jgosmann · 2016-06-23T17:59:56Z

Does it work if you execute jupyter nbextension enable --py --sys-prefix widgetsnbextension and restart the notebook server?

drasmuss · 2016-06-23T18:13:42Z

That command gives ImportError: No module named 'widgetsnbextension'

jgosmann · 2016-06-23T18:17:31Z

Install widgetsnbextension. Then try if it works. If not try the jupyter nbextension ... command and see if that then works.

drasmuss · 2016-06-23T18:21:20Z

Still doesn't work, but new error now:

[IPKernelApp] ERROR | No such comm target registered: jupyter.widget.version
[IPKernelApp] ERROR | No such comm target registered: jupyter.widget.version

jgosmann · 2016-06-23T18:48:02Z

I can't reproduce the problem (using an Anaconda install on Windows 8). How did you install your Python etc?

drasmuss · 2016-06-23T18:52:18Z

All just standard virtualenv setup through Anaconda. It's Windows 10, but I'd be surprised if that made a difference.

drasmuss · 2016-06-23T19:02:39Z

Here's the steps that reproduce the error for me:

> conda create -n tmp2 python=3.5 numpy jupyter matplotlib
> activate tmp2
> python setup.py develop  # (installing nengo)
> pip install widgetsnbextension
> jupyter nbextension enable --py --sys-prefix widgetsnbextension
> jupyter notebook

hunse · 2016-06-23T19:02:39Z

Can we pull this out of 2.1.1? I'd like to get that released, since it helps me with some Nengo OCL stuff a lot.

jgosmann · 2016-06-23T19:05:36Z

Might as well put it in and do another fix in a later release?
I think this should fix most problems, just not all problems. Or is the progress bar working with master for @drasmuss ?

drasmuss · 2016-06-23T19:07:30Z

No, doesn't work in master either.

hunse · 2016-06-23T19:10:34Z

Ok, yeah, that makes sense to put it in then.

jgosmann · 2016-06-23T19:10:40Z

@drasmuss: Can you update ipywidgets to version >= 5?

drasmuss · 2016-06-23T19:27:22Z

That did the trick, works with ipywidgets 5. Is there a reason that the standard jupyter install uses version 4 instead of 5, or just waiting on them to update their requirements?

Anyway, I think this is good to merge, as it's at least >= to the behaviour in master. Just two possible comments come to mind:

Should we add widgetsnbextension and ipywidgets>=5 to requirements-optional.txt? I'm not really clear whether these are actually requirements, or just requirements on my system.
Is it possible to automate whatever jupyter nbextension enable --py --sys-prefix widgetsnbextension is doing?

drasmuss · 2016-06-23T19:33:54Z

Actually it looks like widgetsnbextension comes with ipywidgets>=5, so ipywidgets>=5 is the only requirement. Here are the working install steps, for posterity:

> conda create -n tmp2 python=3.5 numpy jupyter matplotlib
> activate tmp2
> python setup.py develop
> pip install ipywidgets --upgrade
> jupyter nbextension enable --py --sys-prefix widgetsnbextension
> jupyter notebook

jgosmann · 2016-06-23T19:34:21Z

I'm not sure if the combination of ipywidgets 4.1.1 and notebook 4.2 is actually supported by Jupyter in any way. The say what notebook version is required by which ipywidgets version, but not the other way around. Anyways, I don't think it is worth making the code more complicated trying to get those particular versions to work. Maybe we can print a warning if we detect these specific version and instruct the user to update ipywidgets.

Is there a reason that the standard jupyter install uses version 4 instead of 5, or just waiting on them to update their requirements?

I will investigate this, but probably they don't state a version and the it fetches the newest version? All my installs end up with version 5 at least.

Should we add widgetsnbextension and ipywidgets>=5 to requirements-optional.txt? I'm not really clear whether these are actually requirements, or just requirements on my system.

These are actual requirements for the IPython progress bar.

Is it possible to automate whatever jupyter nbextension enable --py --sys-prefix widgetsnbextension is doing?

I don't think so because otherwise Jupyter would already do it? At least it usually prints a warning telling you to execute that command (given you versions match up). Also, if you install jupyter via a package manager this tends to happen automatically by means of the package manager (but that doesn't apply to pip installs like in a virtualenv).

drasmuss · 2016-06-23T19:46:48Z

I will investigate this, but probably they don't state a version and the it fetches the newest version? All my installs end up with version 5 at least.

Not mine

> conda create -n tmp2 python=3.5 jupyter
> activate tmp2
> pip show ipywidgets
---
Metadata-Version: 1.1
Name: ipywidgets
Version: 4.1.1
...

tbekolay · 2016-06-23T20:32:27Z

Seems like conda's jupyter recipe is lagging behind jupyter... I'm inclined not to add anything to requirements as it won't be necessary once conda updates its recipes. Also, worst case is just no progress bar, nothing breaks. So... okay for me to merge then?

jgosmann · 2016-06-23T20:51:41Z

The [Anaconda package list] still lists notebook 4.1.1. Maybe @drasmuss somehow updated notebook without updating ipywidgets?

drasmuss · 2016-06-23T20:53:26Z

You can see all the steps I'm following above, I promise I didn't leave anything out. It's just the standard anaconda jupyter installation.

tbekolay · 2016-06-23T20:55:10Z

The current ipywidgets recipe; note the 4.1.1.

jgosmann · 2016-06-23T20:57:44Z

Indeed, should we create an upstream issue?

tbekolay · 2016-06-23T21:56:53Z

Will merge when @jgosmann removes his assignment 👍 (squash commit looks good to me btw)

jgosmann · 2016-06-23T22:09:35Z

Sorry, forgot to unassign myself. Go ahead. :)

The extension will warn when incompatible notebook and ipywidgets versions are installed. Fixes #1085. Addresses #1084.

jgosmann added the bug label Jun 10, 2016

jgosmann added this to the 2.1.1 release milestone Jun 10, 2016

jgosmann force-pushed the fix-ipy-progress branch from f963e26 to a3a1fe9 Compare June 10, 2016 21:07

jgosmann added the needs review label Jun 10, 2016

jgosmann mentioned this pull request Jun 11, 2016

2.1.1, 2.2.0 releases #1080

Closed

12 tasks

tbekolay self-assigned this Jun 11, 2016

tbekolay removed their assignment Jun 22, 2016

tbekolay mentioned this pull request Jun 23, 2016

Graceful fail when cache index cannot be loaded #1097

Closed

jgosmann self-assigned this Jun 23, 2016

jgosmann removed their assignment Jun 23, 2016

Fix IPython progress bar in notebook versions >= 4

c0643d5

The extension will warn when incompatible notebook and ipywidgets versions are installed. Fixes #1085. Addresses #1084.

tbekolay force-pushed the fix-ipy-progress branch from 428487a to c0643d5 Compare June 23, 2016 22:46

tbekolay merged commit c0643d5 into master Jun 23, 2016

tbekolay deleted the fix-ipy-progress branch June 23, 2016 22:53

tbekolay added reviewed and removed needs review labels Aug 8, 2016

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix IPython progress bar in notebook versions >= 4 #1088

Fix IPython progress bar in notebook versions >= 4 #1088

jgosmann commented Jun 10, 2016 •

edited

Loading

jgosmann commented Jun 10, 2016

tbekolay commented Jun 22, 2016

drasmuss commented Jun 23, 2016 •

edited

Loading

jgosmann commented Jun 23, 2016

drasmuss commented Jun 23, 2016

jgosmann commented Jun 23, 2016

drasmuss commented Jun 23, 2016

jgosmann commented Jun 23, 2016

drasmuss commented Jun 23, 2016

jgosmann commented Jun 23, 2016

drasmuss commented Jun 23, 2016

drasmuss commented Jun 23, 2016 •

edited

Loading

hunse commented Jun 23, 2016

jgosmann commented Jun 23, 2016

drasmuss commented Jun 23, 2016

hunse commented Jun 23, 2016

jgosmann commented Jun 23, 2016

drasmuss commented Jun 23, 2016

drasmuss commented Jun 23, 2016

jgosmann commented Jun 23, 2016

drasmuss commented Jun 23, 2016

tbekolay commented Jun 23, 2016 •

edited

Loading

jgosmann commented Jun 23, 2016

drasmuss commented Jun 23, 2016

tbekolay commented Jun 23, 2016

jgosmann commented Jun 23, 2016

tbekolay commented Jun 23, 2016

jgosmann commented Jun 23, 2016

Fix IPython progress bar in notebook versions >= 4 #1088

Fix IPython progress bar in notebook versions >= 4 #1088

Conversation

jgosmann commented Jun 10, 2016 • edited Loading

jgosmann commented Jun 10, 2016

tbekolay commented Jun 22, 2016

drasmuss commented Jun 23, 2016 • edited Loading

jgosmann commented Jun 23, 2016

drasmuss commented Jun 23, 2016

jgosmann commented Jun 23, 2016

drasmuss commented Jun 23, 2016

jgosmann commented Jun 23, 2016

drasmuss commented Jun 23, 2016

jgosmann commented Jun 23, 2016

drasmuss commented Jun 23, 2016

drasmuss commented Jun 23, 2016 • edited Loading

hunse commented Jun 23, 2016

jgosmann commented Jun 23, 2016

drasmuss commented Jun 23, 2016

hunse commented Jun 23, 2016

jgosmann commented Jun 23, 2016

drasmuss commented Jun 23, 2016

drasmuss commented Jun 23, 2016

jgosmann commented Jun 23, 2016

drasmuss commented Jun 23, 2016

tbekolay commented Jun 23, 2016 • edited Loading

jgosmann commented Jun 23, 2016

drasmuss commented Jun 23, 2016

tbekolay commented Jun 23, 2016

jgosmann commented Jun 23, 2016

tbekolay commented Jun 23, 2016

jgosmann commented Jun 23, 2016

jgosmann commented Jun 10, 2016 •

edited

Loading

drasmuss commented Jun 23, 2016 •

edited

Loading

drasmuss commented Jun 23, 2016 •

edited

Loading

tbekolay commented Jun 23, 2016 •

edited

Loading