# Liquid tags #21

Merged
merged 24 commits into from Aug 28, 2013
+748 −1

## Conversation

Projects
None yet
Contributor

### jakevdp commented May 3, 2013

 This adds a set of extensions to use Liquid-style tags within markdown (similar to those provided by octopress). There are four tags included at this time, though the module is written to be easily extensible: Insert images of a given size: {% img /url/to/image.png [width] [height] [title] [alt]%}  Insert HTML5/flash compatible videos: {% video /url/to/video.mp4 [width] [height] [title] %}  Insert code from a file, with a title and link to download: {% include_code filename [title] %}  Insert an HTML-rendered ipython notebook: {% notebook filename.ipynb [cells[start:end]] %}  Examples and details of how to use each are in the Readme file, and in the doc strings of each plugin.

### jakevdp added some commits May 2, 2013

 initial commit: img, video, include_code tags 
 57d62ce 
 add notebook tag 
 1ae1833 
 add ~ files to gitignore 
 87f81ab 
 fix notebook formatting 
 a934cde 
 make code & notebook directories configurable 
 790fe92 
 update readme 
 a6ad0e6 
 change to work with new nbconvert 
 c0e7562 
 add ablility to specify notebook cells 
 d5e4d17 
 add ability to specify static directory 
 d091f27 
 save notebook header only once 
 e0e303a 
 fix cell indexing issue 
 76d27c2 

Merged

### jakevdp added some commits May 6, 2013

 fix notebook cell parsing 
 5977160 
 add literal tag for displaying {% ... %} 
 ce11ec8 
 properly set language in include_code 
 92e4483 
Member

### almet commented May 17, 2013

 that looks great. However, I would love to have this not only bound to markdown, but also available for people using restructured text, do you think that would be useful?
Contributor

### jakevdp commented May 18, 2013

 I agree that would be nice, but would basically require writing and testing an entirely new implementation. The current one is built on markdown preprocessors, and you'd have to do the equivalent with ReST preprocessors as well. I've never dug into ReST - do you know if it would be straightforward to do?

### mlgill commented May 24, 2013

 When I attempted to used the Liquid tags notebook plugin from this pull request (21) with a very simple markdown post and IPython notebook, I get the following error upon running pelican: WARNING: Could not process /Users/mlgill/Dropbox/themodernscientist/content/2013-05-24-testing_ipython_notebook.md parsing error: didn't find the end of the div  If I comment out lines 144-145 of notebook.py and rerun pelican, everything is fine and the html post looks as expected. Thus, I believe the error is not valid. I have only briefly looked into debugging the issue and haven't yet figured out the problem. Here is some information about my setup: * IPython version 0.13.2 * Liquid tags from pull request 21 of pelican-plugins * nbconvert github commit 0fababc961cbe0b58df4cf0286e6b2c41fb695c53 * nbconvert requires style.min.css to be placed in IPython/frontend/html/notebook/static/css and IPython version 0.13.2 doesn't have this file, so I've tried this using style.min.css from github commit c751c59af3ed736332cf6e2f61d27e39ccc5a788. However, the issue persists when using notebook.css from IPython version 0.13.2 instead. * Pelican version 3.2.1 * Python version 2.7.5 * Mac OS X version 10.8.3  Please let me know if I can provide further information or test something.
Contributor

### jakevdp commented May 24, 2013

 That piece is a bit of an ugly hack -- once the nbconvert architecture is stabilized, we should be able to select cells within nbconvert, rather than slicing up the generated html. It would be better to try and fix this once the nbconvert refactor is finished.

### mlgill commented May 25, 2013

 No problem. I knew what this code was doing and this is why I didn't spend tons of time debugging it since I could myself hack it and make things work. Given the situation with nbconvert, I figured you'd want to wait. At least it's documented now as having been an issue for the future.
 make compatible with newer IPython 
 5af2ff3 

### ibayer commented Jun 13, 2013

 Hey, I'm quit interested in this plug in. Do I need to expect significant changes before this PR gets merged? @jakevdp Thanks for your work!
Member

### justinmayer commented Jun 13, 2013

 Thanks for the contribution, Jake. To recap, here's where we are so far: @ametaireau said: I would love to have this not only bound to Markdown, but also available for people using reStructuredText, do you think that would be useful? @jakevdp replied: I agree that would be nice, but would basically require writing and testing an entirely new implementation. The current one is built on Markdown preprocessors, and you'd have to do the equivalent with reST preprocessors as well. I've never dug into reST - do you know if it would be straightforward to do? I agree reST support would be nice, but I suspect we should merge this in its current form for the following reasons: It's a plugin (not core) and thus perhaps can be afforded more "leeway" Implementing support for reST may be a non-trivial task Nobody has stepped forward to implement equivalent support for reST Last but not least, even if we merge this plugin in its current form, anybody is of course free to pick this up again in the future to implement support for reST, Asciidoc, and other formats. Just my two cents. Any other thoughts?
Contributor

### jakevdp commented Jun 13, 2013

 Actually, I'd like to first make sure the notebook piece works with the finished nbconvert refactor -- I haven't had a chance to do that piece yet. While we're at it, we may as well wait until nbconvert is merged into IPython for the 1.0 release, slated for July.
Member

### almet commented Jun 28, 2013

 Any update?
Contributor

### jakevdp commented Jun 28, 2013

 Not yet - I'm still planning to wait for the IPython 1.0 release, otherwise we'll have to rewrite this code again in a month.
Contributor

### jakevdp commented Jul 3, 2013

 Just so folks can keep up-to-date -- I've been working on getting the features into IPython 1.0 that will allow this PR to be finished, and finished well. IPython work here:

### mlgill commented Jul 3, 2013

 @jakevdp Thanks for the info. Have been waiting patiently for all the necessary updates. Happy to give it a try when ready.

### ibayer commented Jul 3, 2013

 @jakevdp thanks for the update. I'm eager to give it a try too.
Member

### almet commented Jul 11, 2013

 Awesome, we're almost there!
 make liquid_tags.notebook compatible with IPython 1.0 
 2c4f75e 
Contributor

### jakevdp commented Jul 16, 2013

 This commit reworks the notebook tag to use the new IPython.nbconvert submodule, which is part of the yet-to-be-released IPython 1.0. This requires a dev install of IPython (two relevant PRs were merged into IPython master earlier today). As long as nothing else changes in IPython before the 1.0 release, I'd say this is good to go. There may be a few adjustments to the IPython stylesheets before that, though, so I'd prefer to wait until IPython 1.0 is finalized before merging this. In the meantime, if anyone here is willing to test and give feedback on the new code, I'd appreciate it!

### Carreau reviewed Jul 17, 2013

liquid_tags/notebook.py
 end = Integer(MAX_NB_CELLS, config=True, help="last cell of notebook to be converted") def __call__(self, nb, resources):

#### Carreau Jul 17, 2013

you can just overwrite call (without dunder) it is just a stupid wrapper in NBconvert for now, but we might add conveniences stuff later (like being sure to pass a deepcopy of nb instead of a reference).

I think for 1.0 all the "Activatable" prefix will be dropped and the base class will support the enable keyword (just FYI).

### Carreau reviewed Jul 17, 2013

liquid_tags/notebook.py

#### Carreau Jul 17, 2013

does this open (and the one line 253) close the file ? (maybe python does automatically if the file descriptor has no more references, i'm just not sure)

#### jakevdp Jul 17, 2013

Contributor

It should, but it's probably better to use a context manager in any case. I'll change that.

### Carreau reviewed Jul 17, 2013

liquid_tags/notebook.py
 if end: end = int(end) else: end = MAX_NB_CELLS

#### Carreau Jul 17, 2013

I think that slicing with None should work and mean 'end':

In [9]: range(10)[5:None]
Out[9]: [5, 6, 7, 8, 9]

#### jakevdp Jul 17, 2013

Contributor

True, but doesn't the IPython Integer traitlet crash on None?

#### Carreau Jul 17, 2013

Hum, that is true... most of Traitlets accept None, but, Not Integer. I'll see if this could be added on IPython side. like IntegerMaybeNone.

#### jakevdp Jul 17, 2013

Contributor

That would be useful. I suppose I could also define a custom traitlet for the purpose of this plugin if you don't think it's general enough to include in IPython. Let me know what you decide.

### almet reviewed Jul 17, 2013

 SYNTAX = '{% img [class name(s)] [http[s]:/]/path/to/image [width [height]] [title text | "title text" ["alt text"]] %}' # Regular expression to match the entire syntax ReImg = re.compile("""(?P\S.*\s+)?(?P(?:https?:\/\/|\/|\S+\/)\S+)(?:\s+(?P\d+))?(?:\s+(?P\d+))?(?P\s+.+)?""")

#### almet Jul 17, 2013

Member

In general, put the constants in full upper case

### almet reviewed Jul 17, 2013

 ReTitleAlt = re.compile("""(?:"|')(?P[^"']+)?(?:"|')\s+(?:"|')(?P<alt>[^"']+)?(?:"|')""") @LiquidTags.register('img')

#### almet Jul 17, 2013

Member

I love the use of the decorator for that.

Member

### almet commented Jul 17, 2013

 i'm re-reading the whole issue. @jakevdp I think adding the same thing for restructured text wouldn't be hard.
 address notebook comments 
 6f8d398 
Contributor

### jakevdp commented Jul 18, 2013

 I think I addressed all the notebook.py comments. @Carreau - I ended up defining a custom traitlet that allows None for the slicing. It's only a few lines, and I think it makes things much cleaner. Thanks for the suggestion!

### Carreau commented Jul 18, 2013

 I think I addressed all the notebook.py comments. @Carreau - I ended up defining a custom traitlet that allows None for the slicing. It's only a few lines, and I think it makes things much cleaner. Thanks for the suggestion! Yes, looks great. I think that since a few hours ago s/ActivatableTransformers/Transformers/g I'll re-read the plugin (or at least the part that I understand)
 ActivatableTransformer -> Transformer 
 2ad2132 
Contributor

### jakevdp commented Jul 18, 2013

 The last commit keeps this compatible with the current IPython master.
Contributor

### ulikoehler commented Jul 23, 2013

 +1, I'd also like to see this merged as soon as possible, reST functionality can be added later, but to me, this plugin provides a significant usage improvement, especially when including ipython notebooks.

### Carreau commented Jul 23, 2013

 We are sprinting on nbconvert this week, so I will sugest waiting for 1.0 release. We kkep you in mind :-)
Contributor

### ulikoehler commented Jul 23, 2013

 Wow, thanks for the quick answer - I'm happy to hear that and looking forward to the results :-)

### Carreau reviewed Jul 23, 2013

 help="last cell of notebook to be converted") def call(self, nb, resources): nbc = deepcopy(nb)

#### Carreau Jul 23, 2013

Will now take care of that for you :-)

#### Carreau Aug 7, 2013

nband resources are now guaranted to be deepcopied before beeing passed to the first transformer, so you shoudl be safe of not deepcopying it yourself if you are concern with performance.

Contributor

### jakevdp commented Aug 6, 2013

 For anyone curious, this currently does not work with the IPython release candidate (some things have been renamed). I'm working on remedying that.
 Change names for compatibility with IPython 1.0alpha 
 499ee98 
Contributor

### jakevdp commented Aug 7, 2013

 Turns out is was a simple name remapping. This should work with the IPython 1.0 release candidate available at this tag: https://github.com/ipython/ipython/tree/1.0.0a1 There are a few nbconvert bug fixes in the process of being backported from master for release. Again, we should wait until IPython 1.0 is final to merge, but I think this is pretty close!
Member

### justinmayer commented Aug 7, 2013

 Fantastic, Jake. Excited for the release!
 modify mathjax to not interfere with normal posts 
 c545c9e 

### Carreau commented Aug 7, 2013

 Oups, sorry for the name conflict, I try to warn you when there are changes that might affect the plugin, but this one slipped through my fingers.
 adjust anim_icon CSS 
 4643ab7 

### pelson reviewed Aug 10, 2013

#### pelson Aug 17, 2013

It'd be nice if this could handle spaces in the filename. Perhaps allowing people to specify the filename in quotes would disambiguate the situation?

#### jakevdp Aug 20, 2013

Contributor

I agree that would be nice, but I think adding this sort of thing would overly-complicate an already extremely dense regular expression. Something like this would work:

FORMAT = re.compile(r"""^(\s+)?(?P<src>\"[^\"]*\"|\'[^\']*\'|\S+)(\s+)?((cells$)(?P<start>-?[0-9]*):(?P<end>-?[0-9]*)($))?(\s+)?\$""")


But then you end up with a string containing quotation marks if they're used, which adds further complicated post-processing. Additionally, a regular expression like this is extremely difficult to read, maintain, and understand, even if you're the person who wrote it!

The best option would be to write a simple modular parsing framework to express these things within the submodule, but I don't really have the bandwidth to take that on right now. Plus, who puts spaces in their filenames anymore? 😄

### gvwilson commented Aug 17, 2013

 This looks cool - will you be adding {% for %} and {% if %}, and if so, can they wrap cells, i.e., can I put: +------------------------------+ | {% if instructor %} | +------------------------------+ | ...a regular Python cell... | +------------------------------+ | ...a Markdown cell... | +------------------------------+ | {% endif %} | +------------------------------+ and have it work as expected? Because I'd use that right away :-)
Contributor

### jakevdp commented Aug 18, 2013

 @gvwilson - that's a bit beyond the scope of this. The liquid tags are within the pelican markdown file, not within the notebook itself, so I'm not sure how that would work in this context.

### Carreau commented Aug 18, 2013

 @gvwilson this would go into nbconvert template/transformers, then in pelican you could do : {% notebook /path/to/notebook.ipynb [ instructor | learner ] %}

### jakevdp added some commits Aug 20, 2013

 fix mathjax script 
 9e416a7 
 provide warning if IPython 2.0-dev is being used 
 e36dae6 
Contributor

### jakevdp commented Aug 20, 2013

 Aside from several feature requests which I likely won't get to, I think this is ready to merge. I've been using it with IPython 1.0 on my blog for about a week now, and all the bugs seem to be worked out. There's certainly going to be work in the future to make it compatible with IPython 2.0, and perhaps add a better parsing framework than the raw regular expressions I'm currently using. Also, adding RST support would be nice, if someone wants to take that on. None of those are blockers, though, and I'd like to finalize what we have.
 explicitly check for IPython version 
 6cd4482 
Contributor

### jakevdp commented Aug 26, 2013

 A user noted that if sphinx is not installed, then the IPython version check was giving a misleading message. I fixed that.

### AlexMikhalev commented Aug 27, 2013

 What is the current status of this pull request?
Contributor

Member

### justinmayer commented Aug 28, 2013

 Thanks to Jake for all the hard work on this plugin, with due appreciation to others who assisted.

### justinmayer added a commit that referenced this pull request Aug 28, 2013

 Merge pull request #21 from jakevdp/liquid_tags 
Add Liquid Tags plugin
 a20ca76 

Merged

### ocefpaf commented on liquid_tags/notebook.py in 6cd4482Sep 9, 2013

 I believe that this comparison should be with a string ('1') instead of an integer (1).
Owner

### jakevdp replied Sep 9, 2013

 Yes, this was fixed here: 1bda804

### ocefpaf replied Sep 9, 2013

 Thanks.

Member

### almet commented Jun 8, 2018

 Please, Avoid publishing passive aggressive comments like that. It's not helpful, to anyone. It's sometimes hard for maintainers to find the time to work on all the issues, especially when it comes to managing plugins and themes. Pelican-plugins was at first thought as a way to catalog all the plugins. If the project is maintained somewhere else it's a good news, not a bad one :-) Big up to anyone who helped on this. And please, avoid being aggressive to people who commit their free time to open source projects. Cheers, Alex … On 07/06/2018 19:40, Natalino Busa wrote: ... fast forward 5 years, still not merged, although it works perfectly. Fortunately, this very useful and inspiring plugin is maintained on a separate repository ... — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#21 (comment)>, or mute the thread .