Update IPython Notebooks to version 4, fix #2485

[tiangolo]

This pull request updates all the IPython Notebooks to version 4, fix #2485

The description of the problem is in issue #2485, I copy here:

When I open the example notebooks I get a notification saying that they are in an older format (v3) and that they could be updated to (v4):

This notebook has been converted from an older notebook format (v3) to the current notebook format (v4). The next time you save this notebook, the current notebook format will be used. Older versions of IPython may not be able to read the new format. To preserve the original version, close the notebook without saving it.

I'm updating ...all the notebooks to the new version in one go... as said by @shelhamer in a previous PR: #2249 (comment)

[longjon]

Okay, thanks. Know any reason why the new notebooks are an order of magnitude more lines than the old ones?

Is there any reason we shouldn't break backwards compatibility? I guess the BVLC devs are mostly on the latest version, so it will be rather unpleasant to maintain v3, and it's probably better not to have any notebooks in v3, so that users will know right away if they need to upgrade. (@shelhamer, is that what you had in mind?)

[tiangolo]

Nope, I'm note sure why the notebooks are bigger. I guess it might have to do with the change from "IPython" to "Jupyter" and being "language agnostic".

References: http://ipython.org/ and https://jupyter.org/

Here's the v4 spec (although I don't find anything obvious from that): https://github.com/ipython/ipython/wiki/IPEP-17:-Notebook-Format-4

---

Here's my situation: I have the latest version of Anaconda (2.2.0), once I open one of the notebooks it starts yelling at me that it has been updated to v4, and that to preserve v3 I should do not save it.

Most IPython notebook installations have auto-save turned on, so, just by opening any notebook, it gets modified and I get "uncommited changes" in the opened files. I guess that those that have new versions of IPython will have those "uncommited changes" too.

---

What I find important is that new notebooks, if made in a new IPython, will be v4, so there would be different versions of notebooks.

More importantly, bug-fixes or modifications in the current notebooks is harder without having a way to save notebooks in v3, as was in #2249.

I know breaking backwards compatibility is not something very desirable. I just want to put this into your consideration and provide a possible PR to use.

[shelhamer]

The requirements.txt should likewise be updated to reflect our change in expectation of the ipython / jupyter version number.

@longjon Yeah, the idea is to align the notebooks with our environment and make it clear to users that v4 is the way to go.

I don't know why these notebooks are so much larger.

[tiangolo]

I updated the requirements.txt file to have ipython>=3.0.0, the oldest version to support v4 notebooks (although the newest IPython version is 3.1).

---

I was curious about the difference in size so I investigated a bit, I did a diff for each file comparing both versions, the main difference is in how images are encoded, in v3 each image is a long string (I assume Base64), in v4 each image is an array (list) of strings (a notebook is basically a JSON file).

The v4 spec says that an image could be a string or an array of strings.

I guess the IPython notebook automatic conversion to v4 prefers an array strings than a long string, so that's what it generates: an array of fixed length strings, formatted with newline characters. So, there's a lot of newline characters: one for each string in each array of each image.

The PR shows there are about 39000 new lines, but it's mainly because of those newline characters.

Actually, the file sizes keep quite comparable. All the v3 notebooks under the examples directory have a total size of 2.9MB, and all the v4 notebooks have a size of 3.3MB.

[shelhamer]

Thanks for the updates and investigation @tiangolo!

[tiangolo]

I'm glad to help!