# ipython/nbconvert

### Subversion checkout URL

You can clone with
or
.

# Jinja Templates#77

Merged
merged 313 commits into from
+29,008 −649

### 8 participants

Owner

Re-write Nbconvert using Jinja Template.

It Is much more easier to tweek and to make inheriting template,
without having to code a class that inherit from Converter.py

look at converter/converter_template.py
and
templates/*.tpl

I considere templates/python.tpl almost done, and which fixes some
strangeness of previous implementation that had inconsistencies in blank lines.

usage

./nbconvert2.py profile ipynb_file

generating a diff with current ipynb references:

diff -U3  tests/ipynbref/IntroNumPy.orig.html <(./nbconvert2.py profile ipynb_file)


I would really like the though of @ellisonbg, @fperez.
Pinging @damianavila which might be interested.

Todo :

• write a preprocessor that extract files
• make the 'extrator' key configurable
• cleanup the makefile that generate jinja-tex template from jinja template
• create a kind of 'profile' which would contain the configuration for each 'format'
• make the filter/preprocessors 'Configurables'
• write preprocessor worksheet/cell/output decorator to simplify the task
• write RST converter
• write blogger converter
• document
• stabilize API
• check header delimiters : {1: '=', 2: '-', 3: '~', 4: '.', 5: '\'', 6: ''}
• write (more) test
• write separated smaller test
• replicate remove_math replace_math step in md2latex / other(?) converter

Maybe/Later PR

• integrate Damián Reveal converter
• people interested in the same converter but to beamer.
• write a preprocessor that inline files
• write Markdown converter
• add a preprocessor that remove non prioritary display data for conversion speed lower memory footprint

To Fix :

• output not monospace in output html
• tex: <obj> rendered as ¡obj¿
Owner

You read our minds! A few weeks ago when I visited Fernando, we talked briefly about the growing complexity of the converters. The number of options for each converter was growing quickly and the problem with the old approach is that each time someone wanted to do something slightly different, they had to hack the converter code. We specifically talked about using templates as a way to solve this problem and I am thrilled with this direction. We need to review this particular implementation still, but overall this is a great direction.

Owner

I now have the HTML converter working with only a few new lines differences with the old one.
I will clean and organise templates, I have some ideas about how to add more reusability/configurability to templates.

Owner
Owner

How does this PR related to your other one where you convert to using Configurable? Can that one be closed? Are both needed?

Owner

Sometimes we need the ability to convert a block of markdown to another format, such as reST or LaTeX. What is your thought on how to handle that? Expose simple function to the templates, such as md_to_rest and md_to_latex?

Owner

How does this PR related to your other one where you convert to using Configurable? Can that one be closed? Are both needed?

This one is mostly orthogonal, but I will need configurability at some point, hence I builded on top of the other.

The other one can be merge independently I will rebase after.

Sometimes we need the ability to convert a block of markdown to another format, such as reST or LaTeX. What is your thought on how to handle that? Expose simple function to the templates, such as md_to_rest and md_to_latex?

Yes, and you can chain those 'filter' (in jinja language)

Some transformation will still need to be done before template like extracting/re-inlining files.

Latex will need a special treatment because it rely on curly bracket. There is doc on jinja on how to do it. It will just require to generate a null_tex.tpl from null.tpl by replacing all { and } by ( and ) and the inheritance tree of tex template will be separate from the html one.

Owner
Owner

Well, what would be great would be to agree on the name on different block, see if they make sens (null.tpl) and see if the way to extend it is understandable enough.

Having configurable PR merge would be already a step forward.
You can try to have a look at how to reconciliate configurable and docstring inheritor.
I wasn't able to figure out.

Also what would be great would be more ipynb test case to be sure if the new version works ok, I was thinking on bringing back the most popular files of nbviewer as references.

Lastly one big PR that will be a lot of work to convert and might conflict would be Damian's RevealJS one.

So my plan is (more or less in order) :

• Finish Metaui/CellTollbar PR.
• Land the slideshow extension for metaUI/celltoolbar
• Merge Damian's PR.
• Finish this one by testing wether the framework is ok by re-implementing Damian's one
• Progressively replace the current converters by the new ones
• Use the new converter in nbviewer.
• Rest.

How does that sound ?

Owner

You can try to see if it is usable.
I can send PR agains my PR with new implemented converter, or more test, or more Jinja 'filter'
let's try to build a list of what need to be done.

OK, I will look into it soon...

Owner

I thinking os splitting null.tpl into multiple files.

full html seem to be really close in behavior to current version.

Only a handfull of newline differences and a few other small stuff.

To quickly compare you can run

clear ;  diff -U3  tests/ipynbref/00_notebook_tour.orig.html <(./runme.py fullhtml tests/ipynbref/00_notebook_tour.orig.ipynb)  | less -S


On files.

I do not understand why you have differences in the css at the header level... I hope some minor differences in the body because of the templating...but in the css (it is not just a copy from static and added them to the header)? Maybe, I am missing some things (this is the my first encounter with jinja).
Btw, I am trying to understand all this stuff to begin to code some of the converters.

Owner

I do not understand why you have differences in the css at the header level...

The css is extracted from the IPython repo. if there are change in it, it probably come from here.
What are the changes you see ?

I hope some minor differences in the body because of the templating...

Newline if not that easy to get right but that shouldn't matter for HTML

but in the css (it is not just a copy from static and added them to the header)?

Depending on the commit, it use to be a copy and past, now it tries to extract it from different places.

Maybe, I am missing some things (this is the my first encounter with jinja).

Me to, it is very powerful, the doc is well done, it only takes time to fully comprehend to see what is possible.

If you see big differences with the above command line, feel free to past your diff, i'll try to see where it cames from.

Owner

This is what I get different in the css block (I saw another differences
in the body but I understand that those are because of the templating...):

http://pastebin.com/YZq0c5Pj

There are some missing colons, margin, changes from . to #
BTW, the final outcome is the same... so there are subtle changes...

Those are because nbconvert is picking up a old non dev version
of IPython.

Try checking out laster master in your IPython repository.

Coming back... if I wanna to implement a converter the steps would be:

1- Generated the appropiate tpl extended from previous one...

Yes.
You can overwright any {% block %} you want, eventually calling {% super %} if needed.
You should have acces to metadata from template using cell.metadata.[...].

2- Inherit from ConverterTemplate class to create my converter...

Not even sure you need to do that.
To start you can "just" use './runme reveal myfile.ipynb'
If you have a reveal.tpl file it will pick it up.

Something else?

Feel free to submit PR against my branch, and ask questions.

Owner

Things like rm_fake should totally be bart of the base converter.

Depending on what need to be done in process, maybe it also should be optional.
almost nothing from the ConverterTemplate is final now, so why you not try to do it,
ping me and we discuss what should inherited and what should not ?

@Carreau, how should we pull/clone your changes to test? I would like to make this work for getpelican/pelican

Owner

@MichaelMartinez
Right now, pull the branch as is.
Create a template in template folder.
Then

./runme.py [templateNameWithoutDotTPL] [path to ipynb file convert]


Nothing is really final now don't heavily rely on name etc...

Feel free to send me modification or ask if you need.

Short view of where I want to go, not sure if it would work.

in template folder

• null.tpl
• python
• master.tpl
• input.tpl
• output.tpl
• ...
• rst
• master.tpl
• input.tpl
• output.tpl
• ...
• html
• master.tpl
• input.tpl
• output.tpl
• ...

each master.tpl inherit from null.tpl and can 'include' all the other tpl it wishes.
So that you can 'mix and match' by just creating a template file that include what you need.
People could then just write block (like input_no_highlight.tpl, or fancy_in_prompt)

right now I made a small test and Jinja throw undefined variable, I'm still wondering why and didn't had time to look at the reason.

referenced this pull request
Closed

### Fixed LaTeX conversion on Windows #82

Owner
Owner

I would have loop through the cell, potentially happending helper metadata ( that exist only for the time of the conversion) like

• slide start
• slide end
• fragment_start
• fragment_end
# if cell n+1 is a slide cell, then we close the section tag here.
for i,cell in enumerate(cells):
cell.metadata.slideshow.helper = 'slide_end'

Then in the template you can overwite any_cell

{% block any_cell %}
if slide_start
< section >
{{super()}}

if slide_end
</section>
{% endblock %)


### second way

You potentially can look at loop helper of jinja

as actually you can append </section><section> before every 'slide' cell except the first one and the last one (note the closing tag first) and treat separately each the first and last iteration :

if is_slide :
is loop.first?
<section>
else
</section><section>
{{ super() }}

if loop.last ?
</section>


Still you seem to be the one that know reveal, so I might be missing something.

Owner

I will play with this recommendations when I get back home and let you
know...

any time :-)

Owner

I've slightly change the structure of the converter.

After fighting with templates and a lot of thinking (and finally a simple solution)... all the implementation is templated, I mean, all the html is written by templates :-)
I have preprocessed the cells to get some new metadata to help me delimit the slides - as you as pointed out before... I will make some changes for the slightly changes you have commited yesterday... but tomorrow, now, time to go to bed...

Owner

Ok, this should now works also for Tex.
And the ./runme.py now also accepts --Class.trait=value flags.

Owner

This does merge cleanly (fast forward even) no idea why github says otherwise.

Great to hear, you have implemented tex support. I will have a look at the new system and maybe I get an idea how this jinja stuff works :)

Owner

@jakobgager tex support is mainly changing jinja delimiters so the compiler is not confused by {and }.
This is still really early stage in dev, but I'll be happy to get feedback.
What is missing now (especially for latex, but for html also) is a way to extract graphic files from notebook as separate files. But except that finishing latex template is just a question of time.

I had a quick look at the new code yesterday and may give some comments (basically for ipynb2latex conversion):

• I guess the extract graphics stuff can be take from the "old" latex converter.
• Is there a way to change the display_priority for the individual converters? -> I strongly suggest to put the latex output before the image type outputs, otherwise you would get images in nearly all of the cases!
• Do you intend to keep the latex_base.tplx converter as it is and create an additional file like with html to handle the treatment of the individual output types?(to be consistent you probably should rename the html files to html_base and html_full)
• As I posted somewhere else (#76), is it really necessary to distinguish between pyout and display style output? (both can contain the same output types!)
Owner
This was referenced
Closed

### A few of bug fixes/enhancements #84

Closed

Owner

We start to have flags from the command line :

see --help-all flag.

ConverterTemplate options
-------------------------
--ConverterTemplate.display_data_priority=<List>
Default: ['html', 'pdf', 'svg', 'latex', 'png', 'jpg', 'jpeg', 'text']
[wrong help due to copy/past]
--ConverterTemplate.extract_figures=<Bool>
Default: False
wether to remove figure data from ipynb and store them in auxiliary
dictionary

Owner

Pinging readers here, few lines, lots of changes.

Instead of 'format' you can now use a 'profile' name on the command line.

look at profile/*.nbcv

## file latex.nbcv
c = get_config()

c.ConverterTemplate.extract_figures=False
c.ConverterTemplate.template_file='latex_base'
c.ConverterTemplate.tex_environement=True

So basically, you put all your option in this, and then issue a
./runme.py [profile name] [file to treat] it does what you expect. And like IPython if you pass extra flag on the command line they will overwrite the one of the profile.

Does of course works if you inherit from ConverterTemplate, with the name of your custom Class as prefix, as long as the properties are Traits.

Sounds great! Hope i find some time this week to test the new profiles.

Owner

One can now write ipynb preprocessors that inherit from configurable and hence get their configuration from the selected profile.

Owner

Add an example on how to write external files like for Latex.

Owner

some fixes for Latex the files should now be extracted in a folder instead as in the same dir as the ipynb, and the Extractig preferences can also be set, so that for example you extract svg preferably and don't try to extract text as it is useless.

BTW, to write the converted file, use the following :

./runme.py latex_base tests/ipynbref/Gun_Data.orig.ipynb --write=True --no-stdout

 Carreau unified figure folder 143d974 Carreau add blogger html profile 945cd80 Carreau move tex skeleton 33773c4 Carreau inherit blogger 655775d Carreau run some more test should yield them for more grained result 37e9b32 Carreau yield tests 7b76e70 Carreau add display_priority_tex in tests 6776ba2 Carreau partial tex fixes c97de43 Carreau fix fileext and html multidisplay 665be41 Carreau html title 938287e Carreau put figures in subkey d63467b Carreau stateless converter 11a8445
Owner

Converter should now be mostly stateless (unless you play some weird stuff with jinja filters)
but you can now instanciate a converter and feed it many file as you like in a for loop.

 Carreau start rst converter 9aba5d3
referenced this pull request
Closed

### Paragraph & math indent related changes for LaTeX #88

 Carreau  create rst converter and fix some bugs 65d8b93
 Carreau fix data priority rst e115b20
Owner

Rst converter done.

I'm looking at the markdown one, I know that markdown is a superset of html, but the current converter look weird.

I'm wondering if

Let's make sure we have pylab again, in case we have restarted the kernel due to the crash demo above

<div class="highlight"><pre><span class="o">%</span><span class="k">pylab</span> <span class="n">inline</span>
</pre></div>

Welcome to pylab, a matplotlib-based Python environment [backend: module://IPython.zmq.pylab.backend_inline].


Is really what should be spit by the current converter.

I would have expected

Let's make sure we have pylab again, in case we have restarted the kernel due to the crash demo above

%pylab inline

Welcome to pylab, a matplotlib-based Python environment [backend:module://IPython.zmq.pylab.backend_inline].


With maybe some delimiters...

Otherwise I didn't had any comments from core dev in ~3weeks on this PR..

 Carreau reveal converter 2af27a7 Carreau extract css from ipython dir 3502a87
Owner

We now have reveal converter !!! (ok, a few css bug, but the harder part is done), I was hoping for a less than 100 lines templates files, but actually 150... not too bad.

Update :
This WILL BREAK on current master of IPython since the introduction of bootstrap.
You have to use a pre- ipython/ipython#2708 for full_html and jinja converter to work.

 Carreau remove debug statement fc873a7

Hey, Mathias.. you are fast... I am finishing my implementation (different from your logic, less code, more logic in the template), but with your jinja template version from some weeks ago... I have to do an update for the last changes, but i do not know if it makes sense to do it now. What do you think?
BTW, I will try to test this one during the weekend...

Owner

Hi @Carreau,
So I've been playing around with the new converter and the old converter. I love that the new converter uses Jinja as an output template system. Great work! I want to start working on a Latex converter that is similar to to Sphinx's latex exporter (like we talked about in the 'Introductions' email.) Should I write a Jinja template that inherits from the "latex_base.tplx" template and go from there? Or should I work on writing my own template from scratch and worry about inheriting from the base template later?

Owner

Hey, Mathias.. you are fast... I am finishing my implementation (different from your logic, less code, more logic in the template), but with your jinja template version from some weeks ago... I have to do an update for the last changes, but i do not know if it makes sense to do it now. What do you think?

You can stil point me to the changes if even they are Ugly I'm curious to see how you did it. My version is still a hack also so we may merge the 2 versions together.

Owner

Great work! I want to start working on a Latex converter that is similar to to Sphinx's latex exporter (like we talked about in the 'Introductions' email.) Should I write a Jinja template that inherits from the "latex_base.tplx" template and go from there? Or should I work on writing my own template from scratch and worry about inheriting from the base template later?

Feel free to modifie latex_base , If you can't do something by inheriting from it that's probably because it has been badly written.

The way I see it, latex_base should provide markup/environment in the right place that are defined as no-op in the header block so you just have to overwrite the environment definition if you inherit. Feel free to open a PR as soon as you start working on it, and don't be afraid to ask question.

commented on an outdated diff
templates/reveal.tpl
 ((14 lines not shown)) +{% endif %} + +{% if cell.metadata.slideshow.open_section %} +
+{%- endif %} +{% if cell.metadata.slideshow.open_subsection %} +
+{%- endif %} +{% if cell.metadata.slideshow.open_fragment %} +
+{% endif %} +{% if cell.metadata.slideshow.slide_type in ['notes'] %} + +{% elif cell.metadata.slideshow.slide_tyle not in ['skip'] %}
commented on an outdated diff
converters/transformers.py
 ((194 lines not shown)) + + def maybe_close_fragment(self): + """return True is already open, false otherwise + and change state to close + """ + if self.fragment_open : + self.fragment_open = False + return True + else : + return False + + def cell_transform(self, cell, other,count): + ctype = cell.metadata.get('slideshow',{}).get('slide_type',None) + if ctype is None : + cell.metadata.slideshow = {} + cell.metadata.slideshow['slide_type'] = None
 damianavila added a note Feb 2, 2013 Again, just reading... but you have to include the slide_type = '-' ... it appears when you have marked a cell with other label, saved the notebook, and later change again this cells to a "none" ('-') cell. I forgot to include this kind of cell in reveal.orig.ipynb, but you can get it easily just make one cell of this reference to label '-' and you will see this string in the metadata slide_type. to join this conversation on GitHub. Already have an account? Sign in to comment
 Carreau fix typo a6ce310
Owner

thanks @damianavila, fixed.

 Carreau create an activatable transformer classes those are no-op by default 43e8d59 Carreau make ExtracfigureTransformer Activable 9ff2557 Carreau allow configurable filters a385df2 Carreau rename runme into nbconvert2 d512641 Carreau remove commented code 4fc1f42 Carreau start writting more references test f63a0db Carreau move html header out e1cc010 Carreau pylint ba7dbab Carreau add a coalesce stream transformer 9f94aa6 Carreau start to modify fig key configurable f267a6e Carreau fix test for coalesce stream, and fig key config 2abd489
was assigned
Owner

Assigning @fperez :-)

 Carreau cleanup entry point 3f0d2f5 damianavila A new implementation of reveal converter with jinja templates. fd4dd96 damianavila Fixed mathjax inline and some not implemented notice. 4abe32a damianavila Fixed path to notes.js because reveal now detect the reveal path to n… …otes html. e9e859c Carreau document 122bcb1 Carreau more doc and future API 0d83c7e Carreau docs 389a4fd Carreau share some global config state f76aadd Carreau pylint 15d546d Carreau all header leven in rst dec581d Carreau deprecation message in old nbconvert.py 0aa907a
 jdfreder Process header cells. Removed unnecessary tabs. 61b3b53 jdfreder Added rawcell formatting Fixed missing new lines between blocks (prevent Jinja block trim) Removed LaTeX comments f463fa5 jdfreder Moved latex comments in to Jinja blocks so they show in the  compile file. Added Jinja macro to make sure input text does not screw up latex. Added simple text output for unkown cells. a73deec jdfreder Updated installation instructions to install working markdown2latex. 1a1f776 jdfreder Fixed markdown 6a40668 jdfreder README Markdown fixes...README.rst 763597d jdfreder Update README.rst Finally fixed readme formatting... e40fa12 jdfreder Changed readme to correct dependency. b535ce1 jdfreder Enabled markdown blocks. 4911964 jdfreder Added instructions on how to test the new template. 30acf25 jdfreder Added markdown support to headers (temporary) f4b421b jdfreder Update README.rst Fixed restructured text. 51a8aae jdfreder Moved header definitions into individual sphinx doc types. Fixed raw text cell so it doesn't apply any formatting to it's output. 1bef040 damianavila Changed name option from slider to reveal 387781e damianavila Added test_reveal in test_references and fixed test_simple. df644ee damianavila Skipped cells are now hided instead of deleted. 3dcc585 Carreau working config 1143ce9 damianavila A new implementation of reveal converter with jinja templates. 931104e Carreau working config d45446d jdfreder Added datetime access to Jinja b0bae6c jdfreder Fixed some weird things that happened after my first git merge conflict experience. 75283d7 jdfreder Fixed typo 0ac2fba jdfreder Fixed typo 0d63fba jdfreder Added Sphinx transformer. TODO: Investigate alternative to hardcoding... de2ef64 jdfreder Fixed: getattr does not work on dictionaries. Instead use dict.get(key,default) Method declaration uses standard dict, not traitlet dict. c959c7c jdfreder Enforce strict Sphinx style on headers and doc. attrib. Temporarly copy-and-pasted standard latex declares to make output work. fdf5273 jdfreder Changed SphinxTransformer to inherit from ActivatableTransformer f1c1cf6 jdfreder Added code to fix the math $in the notebook so it works with markdown2latex. 328a2ce jdfreder Updated readme to reflect changes in procedure. 26da26e jdfreder PEP8 866dcf3 jdfreder Unecessary re-coding, inherits from base now. 54dd348 jdfreder Added pygments syntax highlighting. Added an HR, need get_lines to bind the HR to the first n lines of text 9677a57 jdfreder Added HR, pygments, made HR attach to first n lines. 819733b Carreau fix issues on rebase 467779f jdfreder Merge pull request #1 from Carreau/fix-9 fix issues on rebase 18d9672 Carreau fix to readme 7e14f62 jdfreder Fixed missing latex_transformer (rebase issue) Added wrapping to template to prevent overflow. cae0fe9 jdfreder Merge pull request #2 from Carreau/fix-9 Backport to readme 2015c19 Carreau Merge pull request #9 from jdfreder/master Sphinx Latex Jinja Template a8af33a Carreau Merge branch master templates Conflicts: README.rst 3622208 Carreau fix enable ebc98ee Carreau fix forgot display priority de5191b Carreau Do now wrap before converting md to latex It can append in the middle of code which breaks this. b500826 Owner \lt does not work in math mode in LaTeX, but does with mathjax. Same with \cases{} which should be replaced with begin/end{cases} Owner Ah, thanks for point this out. Where does \lt and \cases{} show up in the Sphinx stuff? Owner Whoops, nvm, I see we are on the main nbconvert repo. (I thought this was my repo) Owner No problem. this is still a PR of mine so not really the main repo. This are the issue I encounter while programming that I just throw here not to forget. Owner Diverse fix/things to improve (mostly detected on sphinx manual) • extract author/info from notebook metadata (define a structure for that) • issue with bold font and math (**$Z$is discrete**) space removed after second $
• wrong coloration after text wrap see here
• output section even if no output
• some antislash stripped $$\frac{1}{N}\sum_{i=0}^N \;C_i \approx E[\; \lambda \; |\; \alpha ] = \frac{1}{\alpha}(frac) • Four underscore are hline in markdown, try not to escape.... Owner The coloration problem (3rd on your list) also raises another question. Should we make note of when lines are wrapped? Marbe a character like this - http://www.fileformat.info/info/unicode/char/21b5/index.htm . I asked @ellisonbg on HipChat, but I think he is still away from keyboard. Owner I'm not even sure we can do smth for coloration like that. it is probably in issue upstream. Personally I would leave the wrapping without anything. Owner Well, I was looking into this problem earlier, and I found that Latex really is of no help here. There isn't anything that we can do to force the text to wrap nicely when formatted. Unfortunately, this means you are right, it's upstream of the Latex; however, we aren't totally helpless. We could add some logic to wrap the Latex after it's been applied. Some sort of latex - intelligent wrap... Just a thought. But we definitely cannot wrap before passing into Pygments, it won't highlight properly. A perfect example is a long comment line that gets wrapped, the preceding # won't get noticed. Owner You can also fix upstream to ...user, showing a limit line on codemirror : "be carefull, do not go beyond this line" something like 80 characters... referenced this pull request Closed ### Add pdf output format #101 referenced this pull request Closed ### Update static html converter with new notebook CSS #103 added some commits  Carreau fix some relative path issues 5334291 Carreau add init.py for nbviewer 0e77e62 Carreau fix absolute import 6bff5e7 Carreau dont fail on sphinx import 3cc2f3b Carreau fix prompt/output hbox/vbox e6a7d8d referenced this pull request in jupyter/nbviewer Closed ### Nbviewer2 #40  Carreau fix when cell do not have metadata a66b28a Owner I've done a read-through, and nothing stands out to me at this point. This PR is so big that I think what's probably best is to merg it as-is, and then start looking at smaller work from here on out. It already seems preferable to current master in pretty much every way. The only thing I can think of left to be answered is the failing travis tests - @Carreau do you know what's up with that? Are the tests not meant to be running on travis, or is it a real failure? added some commits  Carreau regen latex example d2ed4cc Carreau remove __init__.py 60d1415 Carreau regen html reference 8207dcd Carreau regen rst 26936e5 Carreau bring back nbconvertapp 1bb7d2b Carreau regen html old nbconvert 1c26a93 Carreau regen html old nbconvert 33a1fd4 Carreau regen html old nbconvert c8122e5 Carreau regen html old nbconvert c1ad882 Owner Some of the faillure are real faillure, working on if. @jdfreder there seem to be an issue in remove math: In[3] from converters import latex_transformer lt = latex_transformer.LatexTransformer() lt.enabled = True lt.remove_math_space('e') Out[3]: '$$'

lt.remove_math_space('$e$abc') -> '$e$bc'

Owner

Yikes, thanks for the heads up.
I'll check it out and report back.
It may have been fixed, in my master branch, I have made a lot of changes since the last merge.

 Carreau add test for transformer latex 2d0f45c Carreau fix failing deprecated test a7361cd Carreau fix remove math space ff7d5c0 Carreau regen tex f7dd205 Carreau fix latex base template : wrap traceback in correct env, add extract jpeg, fix coalesce stream add data_latex ad1df49 Carreau fix write binary and non math $ remove write binary for normal write forgott to append text when multiline dollars 994bca5 Carreau regenerate tex references f3ad15c Carreau remove debug statement dcc3fbe Carreau fix latex_transformer and add test b84afae Owner Ok, I fixed the issues I saw, added test, regenerate the templates and quickly checked the diff of the different references templates. so, locally on my machine, commit b84afae: $ nosetests
.........................../Users/bussonniermatthias/nbconvert/tests/tutorial.rst.ref:27: (ERROR/3) Unknown interpreted text role "ref".
/Users/bussonniermatthias/nbconvert/tests/tutorial.rst.ref:48: (ERROR/3) Unknown interpreted text role "ref".
/Users/bussonniermatthias/nbconvert/tests/tutorial.rst.ref:84: (ERROR/3) Unknown interpreted text role "ref".
/Users/bussonniermatthias/nbconvert/tests/tutorial.rst.ref:127: (ERROR/3) Unknown interpreted text role "ref".
/Users/bussonniermatthias/nbconvert/tests/tutorial.rst.ref:157: (ERROR/3) Unknown interpreted text role "ref".
/Users/bussonniermatthias/nbconvert/tests/tutorial.rst.ref:157: (ERROR/3) Unknown interpreted text role "ref".
/Users/bussonniermatthias/nbconvert/tests/tutorial.rst.ref:190: (ERROR/3) Unknown interpreted text role "ref".
............................
----------------------------------------------------------------------
Ran 55 tests in 10.800s

OK

Owner

Any one to give a last green light to merge or even press the big grey button ?

Owner
 Carreau regen references with last master da3916c Carreau fix import error with new docutils d5b6464
Owner

@minrk can you try again ?

Owner

docutils.math moved to docutils.utils.math. after fixing that, there's just one failure:

======================================================================
FAIL: test_templates.test_evens('rst', 'tests/ipynbref/IntroNumPy.orig.ipynb', 'tests/template_ref/IntroNumPy_orig.rst')
----------------------------------------------------------------------
Traceback (most recent call last):
File "/Users/minrk/Library/Python/2.7/lib/python/site-packages/nose/case.py", line 197, in runTest
self.test(*self.arg)
File "/Users/minrk/dev/ip/nbconvert/tests/test_templates.py", line 54, in test_profile
File "/Users/minrk/dev/ip/nbconvert/tests/test_templates.py", line 17, in compfiles
cleanfile(strb))
AssertionError: Lists differ: [u'An Introduction to the Scie... != [u'An Introduction to the Scie...

First differing element 130:
integrated between :math:a=1 and :math:b=9.
integrated between :math:a=1 and :math:b=9\ .

Diff is 71038 characters long. Set self.maxDiff to None to see it.

 Carreau fix docutils math import error on newer version ff4dd7b
Owner

docutils.math stuff should be fixed.

Last test depends on pandoc version version. I downgraded mine to be as close as possible from Travis one.
If you achieve to build references test that also passes on travis, that would be great.
Personally I gave up trying to have test passing on multiple computer.

Owner

Personally I gave up trying to have test passing on multiple computer.

Tests that only pass on one machine aren't exactly tests.

But anyway, I think we are good enough at this point, since nbconvert has never really been tested, we can't call it a regression. I'm going to merge now, so we have a more sensible master.

merged commit fae0063` into ipython:master
Owner

Great ! @jdfreder I missed you message about your PR. Yes, you can open it against master now, all IPython core dev will have access to it now.

Yes, finally landed... congratulations for your great work @Carreau... and all who contribute with code and reviews.