Skip to content

Loading…

WIP: Sphinx Notebook Extension #35

Merged
merged 4 commits into from

4 participants

@jakevdp

Description

This is a quick-and-dirty sphinx extension to include notebooks in sphinx documents. It's based on the raw html directive, so will only work with html.

Usage

in your sphinx directory, add to conf.py:

sys.path.insert(0, '/path/to/nbcovert/')
extensions = ['notebook_sphinxext']

Then in your sphinx document, use:

.. notebook:: path/to/notebook.ipynb

Then type make html, and you should see your notebook embedded in your sphinx document.

Problems

There are a few problems with this implementation... some will be fairly easy to fix, while some need some ipython/sphinx expertise that I don't have.

  • since this is based on the raw directive, it will only work in html, not latex or other outputs. Fixing this will involve diving deeper into sphinx, and also writing a notebook to LaTeX converter
  • currently the CSS is implemented using an HTML5 scoped style tag: this may be a problem in older browsers (see discussion below)
  • it would be nice to add a flag which optionally adds a link to the notebook file for download.
  • I'm sure there are more issues... someone with more sphinx experience could probably help out!
@jakevdp

The biggest problem I'm trying to figure out right now is how to get the CSS into the correct place in the generated document. I sent the question to sphinx-dev, and am waiting for a response.

@jakevdp

Here's the thread on sphinx-dev: https://groups.google.com/forum/?fromgroups=#!topic/sphinx-dev/iG0fB9-JdNM

It looks like the users will have to use the extra step of creating a custom HTML theme to enable notebook includes.

Some other potential options are listed here: http://stackoverflow.com/questions/4606708/declare-css-style-outside-the-head-element-of-an-html-page

Or, we could write a version of the HTML converter that uses all the CSS in-line, rather than in a style sheet.

@jakevdp

OK - I think that using HTML5's scoped style attribute is the most elegant solution here: http://www.w3schools.com/html5/att_style_scoped.asp

This would involve creating a <div></div> around the notebook html, with the css style tags within the div. The problem is that it may not render correctly on older browsers.

@jakevdp

The last two commits clean up the code and use the HTML5 scoped style attribute to implement the notebook.
Note that scoped is not supported by many browsers, but most seem to handle <style> tags in the body of the document without a problem.

Also, I've removed the temporary html file and done everything in memory.

@Carreau

Hi jake,
Sorry for the silence, we are not forgetting you,
Right now i'm trying to decrease the number of open PR on the main IPython repository. But it is really great to be able to have sphinx docs !

Thanks !

@Carreau

Hi,
I don't have a good knowlegde of sphinx, but this look ok to me.
(link to example of what can be done)

As an alternative to scope css, we could prefix all the css selector with div.ipynotebook
so that the style only applies to the element inside the ipynotebook div.

Would it help ?

IMHO we should merge all current open PR on nbconvert, then refuse any PR until the current nbconvert.py is splitted into multiple files, each containing a converter.

After we can start refactoring piece by piece and decide what direction we go into (OO or functionnal)

I would really like to do that because nbviewer embeded version of nbconvert start to really diverge from this one.

What do other @ipython dev think ?

@ellisonbg
IPython member
@Carreau

i am not quite sure what you are asking here. Do you think we need to add
a fixed prefix to all of our css classes in the notebook, like "ipynb_"?
Is that causing problems wrt the Sphinx extension development?

no, i'm suggesting changing css from (table as an example)

table {smth}

to

.ipynotebook table {the same thing}

so that the css only apply to a sub-part of the html, which is nice for embedding.

It is really easy to do with sass or less :

.ipython {
      table {the same thing}
      .....
}

will compile to what you want.

@ellisonbg
IPython member
@Carreau

Ah yes, that will solve some of the problems. But what if another library
uses css class names that we use. Won't their styles be applied to our
classes? I think your solutions will only prevent out css styles from
being applied to their tags, not the other way around.

Depends in which order css is put in the header.

Then if people want to use our css, they are assured it won't clash with already in place css.
It they need something more complex, with different kind of css we can assume they ave acces to css order/html.

In any case, it improves the situation.

@Carreau

Hum, I receive a responce of @jakevdp by mail but it does not appear here.

Obviously in python we can add a (configurable) suffix to all class names, even if it will be painfull, because I think we will try to keep as much css in common between nbconvert and the notebook.

I don't know if it is easily doable with less/sass.

@jakevdp

I wrote something, but it's gone! Weird.

I wrote a bit about my experiences in http://jakevdp.github.com/blog/2012/10/04/blogging-with-ipython/
The problem was that the notebook CSS over-rode the style sheets from the blog, and messed up the whole page. I had to do some changes seen in the hackish script on the page to remove duplicate tags. I think the resulting tags look alright, and are pretty unlikely to interfere with site's default style sheets.

@Carreau

Merging tomorrow if no objection, after I start splitting nbconvert files.

@jakevdp

Sounds good to me. Eventually I'd love to create an extension which converts the internal notebook layout directly into the internal sphinx layout, so that html, latex, etc. works automatically. But that's a bigger project.

@Carreau Carreau merged commit aeee080 into ipython:master
@cdeil cdeil referenced this pull request in pyfit/pyfit
Open

How to generate documentation? #4

@cdeil cdeil referenced this pull request in astropy/astropy
Closed

IPython notebooks for astropy tutorials #559

@megies

It seems to me that in the process of moving/merging this directly into IPython the Sphinx extension has gone under. Am I right and this should be fixed? Or is it not feasible or useful anymore? Or am I missing something here completely (I grepped for "Directive" in the current IPython master and only found the ipython Directive for sphinx)?

@Carreau

Yes, lots of things in nbconvert have "disappeared" in the merging process, as they will most probably be done from scratch. We are still refining nbconvert api before tackling other project that will rely on it. But we want to have most of our doc written as notebooks. This might goes through a sphinx extension.

@megies

Ok, I think I'll try to adapt the old sphinx extension code locally and get it to run with the current IPython.nbconvert code base as an interim solution.

@Carreau

Nbconvert has been re-written almost from scratch. Adapting will probably be harder than re-writing.

@megies

Hmm, looking at the old extension code I think the only use of nbconvert is here, which is getting a html string for a given notebook in the file system. I understand I just need to figure out how to get the html string (from nbconvert.exporters.html.HTMLExporter I guess) with the new code base.

@Carreau

But manipulating the html after nbconvert step will be considered bad practice as nbconvert now support templates.
So all the part of encapsulating by appending stuff and searching for <style and replacing it should be re-written using the templates.

Note that one might also want to convert notebook to rst and then run Sphinx on those notebooks.

@jdfreder jdfreder referenced this pull request in ipython/ipython
Open

Documentation on notebook sphinx integration #4936

@jasongrout jasongrout pushed a commit that referenced this pull request
@bollwyvl bollwyvl two passes through cells
fixes #35
abbac46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Showing with 93 additions and 3 deletions.
  1. +6 −3 nbconvert.py
  2. +87 −0 notebook_sphinxext.py
View
9 nbconvert.py
@@ -272,6 +272,11 @@ def dispatch_display_format(self, format):
def convert(self, cell_separator='\n'):
lines = []
lines.extend(self.optional_header())
+ lines.extend(self.main_body(cell_separator))
+ lines.extend(self.optional_footer())
+ return u'\n'.join(lines)
+
+ def main_body(self, cell_separator='\n'):
converted_cells = []
for worksheet in self.nb.worksheets:
for cell in worksheet.cells:
@@ -281,9 +286,7 @@ def convert(self, cell_separator='\n'):
remove_fake_files_url(cell)
converted_cells.append('\n'.join(conv_fn(cell)))
cell_lines = cell_separator.join(converted_cells).split('\n')
- lines.extend(cell_lines)
- lines.extend(self.optional_footer())
- return u'\n'.join(lines)
+ return cell_lines
def render(self):
"read, convert, and save self.infile"
View
87 notebook_sphinxext.py
@@ -0,0 +1,87 @@
+import sys
+import os.path
+import re
+import time
+from docutils import io, nodes, statemachine, utils
+from docutils.error_reporting import ErrorString
+from docutils.parsers.rst import Directive, convert_directive_function
+from docutils.parsers.rst import directives, roles, states
+from docutils.parsers.rst.roles import set_classes
+from docutils.transforms import misc
+
+from nbconvert import ConverterHTML
+
+
+class Notebook(Directive):
+ """
+ Use nbconvert to insert a notebook into the environment.
+ This is based on the Raw directive in docutils
+ """
+ required_arguments = 1
+ optional_arguments = 0
+ final_argument_whitespace = True
+ option_spec = {}
+ has_content = False
+
+ def run(self):
+ # check if raw html is supported
+ if not self.state.document.settings.raw_enabled:
+ raise self.warning('"%s" directive disabled.' % self.name)
+
+ # set up encoding
+ attributes = {'format': 'html'}
+ encoding = self.options.get(
+ 'encoding', self.state.document.settings.input_encoding)
+ e_handler = self.state.document.settings.input_encoding_error_handler
+
+ # get path to notebook
+ source_dir = os.path.dirname(
+ os.path.abspath(self.state.document.current_source))
+ nb_path = os.path.normpath(os.path.join(source_dir,
+ self.arguments[0]))
+ nb_path = utils.relative_path(None, nb_path)
+
+ # convert notebook to html
+ converter = ConverterHTML(nb_path)
+ converter.read()
+
+ # add HTML5 scoped attribute to header style tags
+ header = map(lambda s: s.replace('<style', '<style scoped="scoped"'),
+ converter.header_body())
+
+ # concatenate raw html lines
+ lines = ['<div class="ipynotebook">']
+ lines.extend(header)
+ lines.extend(converter.main_body())
+ lines.append('</div>')
+ text = '\n'.join(lines)
+
+ # add dependency
+ self.state.document.settings.record_dependencies.add(nb_path)
+ attributes['source'] = nb_path
+
+ # create notebook node
+ nb_node = notebook('', text, **attributes)
+ (nb_node.source, nb_node.line) = \
+ self.state_machine.get_source_and_line(self.lineno)
+
+ return [nb_node]
+
+
+class notebook(nodes.raw):
+ pass
+
+
+def visit_notebook_node(self, node):
+ self.visit_raw(node)
+
+
+def depart_notebook_node(self, node):
+ self.depart_raw(node)
+
+
+def setup(app):
+ app.add_node(notebook,
+ html=(visit_notebook_node, depart_notebook_node))
+
+ app.add_directive('notebook', Notebook)
Something went wrong with that request. Please try again.