New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixes #720 : Auto generate ipython notebooks #1022

Open
wants to merge 5 commits into
base: master
from

Conversation

Projects
None yet
5 participants
@sahmed95
Contributor

sahmed95 commented Mar 26, 2016

Simple Hack to get ipython notebooks out of examples. Needs a little work with markup and equations but other wise should be fine.

@arokem Check sample here

@sahmed95 sahmed95 force-pushed the sahmed95:issue-720_notebooks branch from 1a5befa to f0d45b5 Apr 6, 2016

@arokem

This comment has been minimized.

Member

arokem commented Apr 6, 2016

This is definitely going in the right direction. Would it be possible to do this as a separate instruction in the doc Makefile, rather than part of running the examples?

https://github.com/nipy/dipy/blob/master/doc/Makefile

I think it will be cleaner that way.

@sahmed95 sahmed95 force-pushed the sahmed95:issue-720_notebooks branch 2 times, most recently from 9880737 to f0d45b5 Apr 10, 2016

@sahmed95

This comment has been minimized.

Contributor

sahmed95 commented Apr 12, 2016

@arokem Sorry for the long intervals, is this what you suggested ? I added the code to make ipython notebooks to the dipy/tools/make_examples.py file and as such whenever the valid python examples are built, the notebooks will be built automatically.

I still have some cleaning up to do but, is this a good way to put the notebook conversion code? Or should I have a separate make_notebooks.py file in tools, which is called in a similar fashion as make_examples.py

@Garyfallidis

This comment has been minimized.

Member

Garyfallidis commented May 21, 2016

@sahmed95 is this stalled for now?

@sahmed95

This comment has been minimized.

Contributor

sahmed95 commented May 22, 2016

Need to find the right place to run the code during a build that auto-generates the Ipython notebooks. Also, need to find a way to implement and update gists as @arokem suggested.

Even better, a script that would automatically create/update a gist on github and create the nbviewer page would be fantastic.

Finally, maybe add the links to download the Ipython notebooks in the examples in tutorials.

Will look into it

@arokem

This comment has been minimized.

Member

arokem commented May 22, 2016

Would be good to see what sphinx-gallery
http://sphinx-gallery.readthedocs.io/en/latest/ does, and see whether we
can use some of that machinery. We can't really use sphinx gallery as-is,
because the format of our examples is quite different (they only allow one
section of text at the top, and we want to interleave them), but we can see
what they do under the hood for the conversion. We can also roll our own
using nbconvert http://nbconvert.readthedocs.io/en/latest/. Automating
that to happen during the doc build should be rather straight-forward.

On Sun, May 22, 2016 at 9:08 AM, Shahnawaz Ahmed notifications@github.com
wrote:

Need to find the right place to run the code during a build that
auto-generates the Ipython notebooks. Also, need to find a way to implement
and updates gists as @arokem https://github.com/Arokem suggested.

Even better, a script that would automatically create/update a gist on
github and create the nbviewer page would be fantastic.

Finally, maybe add the links to download the Ipython notebooks in the
examples in tutorials.

Will look into it


You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#1022 (comment)

Shahnawaz Ahmed added some commits Jun 1, 2016

@sahmed95 sahmed95 force-pushed the sahmed95:issue-720_notebooks branch from d5cb8ec to 53b5bd5 Jun 3, 2016

@sahmed95

This comment has been minimized.

Contributor

sahmed95 commented Jun 3, 2016

This is languishing as I haven't really worked with Sphinx. I separated the code to generate Ipython notebooks from the standard examples. The function is documented now.

read_example gets the text from a valid example in doc/examples/ and make_notebook converts it into a notebook which can be written using write_notebook

@arokem

This comment has been minimized.

Member

arokem commented Jun 5, 2016

I think it would make sense to integrate a call to this into the Makefile, in a similar manner to the way that things currently work for converting from .py to .rst (https://github.com/nipy/dipy/blob/master/doc/Makefile#L122), so that when the doc html pages are generated, they include a link to the generated notebook file. Does that make sense?

@sahmed95

This comment has been minimized.

Contributor

sahmed95 commented Jun 5, 2016

Yes. Even I initially started towards that and had put the code inside the make_examples.py file. I can put a call there to make_notebooks.py which will build the examples. So where do we put the notebooks (as a separate folder in docs "example_notebooks") I couldn't follow your last suggestion about Sphinx and nbconvert @arokem. Since dipy doesn't use readthedocs.

Do you suggest that we have our own readthedocs and publish examples such as :
http://sphinx-gallery.readthedocs.io/en/latest/auto_examples/index.html

Similarly, http://nbconvert.readthedocs.io/en/latest/nbconvert_library.html#Extracting-Figures-using-the-RST-Exporter also looks great.

I could setup a readthedocs for Dipy following : http://read-the-docs.readthedocs.io/en/latest/getting_started.html#in-rst

I am in favour of nbconvert rather than sphinx-gallery since it also has interleaved text.

@arokem

This comment has been minimized.

Member

arokem commented Jun 5, 2016

No - I don't think we want to set up readthedocs for this. For two reasons: the first is that our documentation will have a hard time building on a free/automated platform like readthedocs, because our examples require large amounts of data and take a long time to run. The second is that @ghoshbishakh is already working on a new version of the website as a GSoC project over here: https://github.com/nipy/dipy_web, so it would be a duplicated effort. For now, all I am suggesting is to automate things so that when we run make html it not only produces the .rst versions of the examples, but also the .ipynb versions. I was pointing to nbconvert because it is a library for converting between different file formats and notebooks, and it might be useful in doing this conversion (instead of the code you wrote here). Might be worth checking it out. Similarly, sphinx_gallery have already built machinery for creating ipython notebooks and integrating them into sphinx documentation, so it might be worth looking what they have done, so we don't need to reinvent the wheel.

@sahmed95

This comment has been minimized.

Contributor

sahmed95 commented Jun 6, 2016

@arokem Thanks for clearing that out. I have the py ->> ipynb conversion script now and added it to the Makefile under "ipynbexamples"

Running "make ipynbexamples" from the /doc folder generates all the valid_examples in the folder "ipython_notebooks". The notebooks aren't executed though and all that this script does is convert the .py files in the "examples" directory to Ipython notebooks. Now, do we convert the notebooks to rst and then add the rst filenames so that make html builds it just like the python examples?

@Garyfallidis Any pointers on where this should go from here ?(Or maybe @ghoshbishakh can take over) Making a ipynb ->> rst conversion seems simple. [http://sphinx-ipynb.readthedocs.io/en/latest/howto.html]

But I 'm not sure where they should be put and how to get them published. A nice way maybe to make an example gallery : http://sphinx-gallery.readthedocs.io/en/latest/auto_examples/just_code.html#sphx-glr-auto-examples-just-code-py
with a download option for getting the notebook or python example.

@sahmed95

This comment has been minimized.

Contributor

sahmed95 commented Feb 17, 2017

Can this be closed now?

@arokem

This comment has been minimized.

Member

arokem commented Feb 17, 2017

Yes. Closing.

@arokem arokem closed this Feb 17, 2017

@arokem arokem reopened this Jun 19, 2018

@pep8speaks

This comment has been minimized.

pep8speaks commented Jun 19, 2018

Hello @sahmed95, Thank you for updating !

Line 39:18: W291 trailing whitespace
Line 63:9: E722 do not use bare except'
Line 83:75: W291 trailing whitespace
Line 159:1: E305 expected 2 blank lines after class or function definition, found 1

@arokem

This comment has been minimized.

Member

arokem commented Jun 19, 2018

We should follow up on this one and finish this up!

@dmreagan

This comment has been minimized.

Contributor

dmreagan commented Jun 19, 2018

It would be cool to host the notebooks on a service like binder or Colaboratory so beginning users can start playing with things in their web browser. I don't think interactive VTK windows are going to happen, but maybe we could show renders.

@arokem

This comment has been minimized.

Member

arokem commented Jun 19, 2018

@Garyfallidis

This comment has been minimized.

Member

Garyfallidis commented Jun 19, 2018

Good idea! :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment