Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Liquid tags #21

Merged
merged 24 commits into from Aug 28, 2013
Merged

Liquid tags #21

merged 24 commits into from Aug 28, 2013

Conversation

@jakevdp
Copy link
Contributor

@jakevdp jakevdp commented May 3, 2013

This adds a set of extensions to use Liquid-style tags within markdown (similar to those provided by octopress). There are four tags included at this time, though the module is written to be easily extensible:

  • Insert images of a given size:

    {% img /url/to/image.png [width] [height] [title] [alt]%}
    
  • Insert HTML5/flash compatible videos:

    {% video /url/to/video.mp4 [width] [height] [title] %}
    
  • Insert code from a file, with a title and link to download:

    {% include_code filename [title] %}
    
  • Insert an HTML-rendered ipython notebook:

    {% notebook filename.ipynb [cells[start:end]] %}
    

Examples and details of how to use each are in the Readme file, and in the doc strings of each plugin.

@almet
Copy link
Member

@almet almet commented May 17, 2013

that looks great.

However, I would love to have this not only bound to markdown, but also available for people using restructured text, do you think that would be useful?

@jakevdp
Copy link
Contributor Author

@jakevdp jakevdp commented May 18, 2013

I agree that would be nice, but would basically require writing and testing an entirely new implementation. The current one is built on markdown preprocessors, and you'd have to do the equivalent with ReST preprocessors as well. I've never dug into ReST - do you know if it would be straightforward to do?

@mlgill
Copy link

@mlgill mlgill commented May 24, 2013

When I attempted to used the Liquid tags notebook plugin from this pull request (21) with a very simple markdown post and IPython notebook, I get the following error upon running pelican:

WARNING: Could not process /Users/mlgill/Dropbox/themodernscientist/content/2013-05-24-testing_ipython_notebook.md
parsing error: didn't find the end of the div

If I comment out lines 144-145 of notebook.py and rerun pelican, everything is fine and the html post looks as expected. Thus, I believe the error is not valid. I have only briefly looked into debugging the issue and haven't yet figured out the problem.

Here is some information about my setup:

* IPython version 0.13.2
* Liquid tags from pull request 21 of pelican-plugins
* nbconvert github commit 0fababc961cbe0b58df4cf0286e6b2c41fb695c53
* nbconvert requires `style.min.css` to be placed in `IPython/frontend/html/notebook/static/css` and IPython version 0.13.2 doesn't have this file, so I've tried this using `style.min.css` from github commit c751c59af3ed736332cf6e2f61d27e39ccc5a788. However, the issue persists when using `notebook.css` from IPython version 0.13.2 instead.
* Pelican version 3.2.1
* Python version 2.7.5
* Mac OS X version 10.8.3

Please let me know if I can provide further information or test something.

@jakevdp
Copy link
Contributor Author

@jakevdp jakevdp commented May 24, 2013

That piece is a bit of an ugly hack -- once the nbconvert architecture is stabilized, we should be able to select cells within nbconvert, rather than slicing up the generated html.

It would be better to try and fix this once the nbconvert refactor is finished.

@mlgill
Copy link

@mlgill mlgill commented May 25, 2013

No problem. I knew what this code was doing and this is why I didn't spend tons of time debugging it since I could myself hack it and make things work. Given the situation with nbconvert, I figured you'd want to wait. At least it's documented now as having been an issue for the future.

@ibayer
Copy link

@ibayer ibayer commented Jun 13, 2013

Hey, I'm quit interested in this plug in. Do I need to expect significant changes before this PR gets merged?
@jakevdp Thanks for your work!

@justinmayer
Copy link
Member

@justinmayer justinmayer commented Jun 13, 2013

Thanks for the contribution, Jake. To recap, here's where we are so far:

@ametaireau said:

I would love to have this not only bound to Markdown, but also available for people using reStructuredText, do you think that would be useful?

@jakevdp replied:

I agree that would be nice, but would basically require writing and testing an entirely new implementation. The current one is built on Markdown preprocessors, and you'd have to do the equivalent with reST preprocessors as well. I've never dug into reST - do you know if it would be straightforward to do?

I agree reST support would be nice, but I suspect we should merge this in its current form for the following reasons:

  1. It's a plugin (not core) and thus perhaps can be afforded more "leeway"
  2. Implementing support for reST may be a non-trivial task
  3. Nobody has stepped forward to implement equivalent support for reST

Last but not least, even if we merge this plugin in its current form, anybody is of course free to pick this up again in the future to implement support for reST, Asciidoc, and other formats.

Just my two cents. Any other thoughts?

@jakevdp
Copy link
Contributor Author

@jakevdp jakevdp commented Jun 13, 2013

Actually, I'd like to first make sure the notebook piece works with the finished nbconvert refactor -- I haven't had a chance to do that piece yet. While we're at it, we may as well wait until nbconvert is merged into IPython for the 1.0 release, slated for July.

@almet
Copy link
Member

@almet almet commented Jun 28, 2013

Any update?

@jakevdp
Copy link
Contributor Author

@jakevdp jakevdp commented Jun 28, 2013

Not yet - I'm still planning to wait for the IPython 1.0 release, otherwise we'll have to rewrite this code again in a month.

@jakevdp
Copy link
Contributor Author

@jakevdp jakevdp commented Jul 3, 2013

Just so folks can keep up-to-date -- I've been working on getting the features into IPython 1.0 that will allow this PR to be finished, and finished well. IPython work here:

@mlgill
Copy link

@mlgill mlgill commented Jul 3, 2013

@jakevdp Thanks for the info. Have been waiting patiently for all the necessary updates. Happy to give it a try when ready.

@ibayer
Copy link

@ibayer ibayer commented Jul 3, 2013

@jakevdp thanks for the update. I'm eager to give it a try too.

@almet
Copy link
Member

@almet almet commented Jul 11, 2013

Awesome, we're almost there!

@jakevdp
Copy link
Contributor Author

@jakevdp jakevdp commented Aug 6, 2013

For anyone curious, this currently does not work with the IPython release candidate (some things have been renamed). I'm working on remedying that.

@jakevdp
Copy link
Contributor Author

@jakevdp jakevdp commented Aug 7, 2013

Turns out is was a simple name remapping. This should work with the IPython 1.0 release candidate available at this tag: https://github.com/ipython/ipython/tree/1.0.0a1

There are a few nbconvert bug fixes in the process of being backported from master for release. Again, we should wait until IPython 1.0 is final to merge, but I think this is pretty close!

@justinmayer
Copy link
Member

@justinmayer justinmayer commented Aug 7, 2013

Fantastic, Jake. Excited for the release!

@Carreau
Copy link

@Carreau Carreau commented Aug 7, 2013

Oups, sorry for the name conflict, I try to warn you when there are changes that might affect the plugin, but this one slipped through my fingers.

- First, the plugin requires that the nbconvert package [1]_ to be in the
python path. For example, in bash, this can be set via

>$ export PYTHONPATH=/path/to/nbconvert/

This comment has been minimized.

@pelson

pelson Aug 10, 2013

Think this needs updating since nbconvert is now in IPython.

Because the notebook relies on some rather extensive custom CSS, the use of
this plugin requires additional CSS to be inserted into the blog theme.
After typing "make html" when using the notebook tag, a file called
``_nb_header.html`` will be produced in the main directory. The content

This comment has been minimized.

@pelson

pelson Aug 10, 2013

Sadly the _nb_header.html didn't quite cover all of the bases for color selection and I ended up having to add .highlight-ipynb .n { color: black; } manually to the header to see certain sections of my code. I'm using the Just-Read theme...

This comment has been minimized.

@jakevdp

jakevdp Aug 15, 2013
Author Contributor

Is this because the default text color is non-black?

This comment has been minimized.

@pelson

pelson Aug 16, 2013

I guess so (It's inherited from the template). What it essentially means is that the css being generated by the pygments call is not sufficient to cover the html which it is generating, which suggests there is a synchronisation issue. I've tried to dig into it, but tbh, its all pretty deep inside IPython and then even deeper in pygments.

@jakevdp
Copy link
Contributor Author

@jakevdp jakevdp commented Aug 15, 2013

We'll need to watch out for this change, though it's IPython 2.0: ipython/ipython#4044

@Carreau
Copy link

@Carreau Carreau commented Aug 16, 2013

We will probably also rename Exporter -> TemplateExporter as we will have one more base classe.

# Below is the pelican plugin code.
#
SYNTAX = "{% notebook /path/to/notebook.ipynb [ cells[start:end] ] %}"
FORMAT = re.compile(r"""^(\s+)?(?P<src>\S+)(\s+)?((cells\[)(?P<start>-?[0-9]*):(?P<end>-?[0-9]*)(\]))?(\s+)?$""")

This comment has been minimized.

@pelson

pelson Aug 17, 2013

It'd be nice if this could handle spaces in the filename. Perhaps allowing people to specify the filename in quotes would disambiguate the situation?

This comment has been minimized.

@jakevdp

jakevdp Aug 20, 2013
Author Contributor

I agree that would be nice, but I think adding this sort of thing would overly-complicate an already extremely dense regular expression. Something like this would work:

FORMAT = re.compile(r"""^(\s+)?(?P<src>\"[^\"]*\"|\'[^\']*\'|\S+)(\s+)?((cells\[)(?P<start>-?[0-9]*):(?P<end>-?[0-9]*)(\]))?(\s+)?$""")

But then you end up with a string containing quotation marks if they're used, which adds further complicated post-processing. Additionally, a regular expression like this is extremely difficult to read, maintain, and understand, even if you're the person who wrote it!

The best option would be to write a simple modular parsing framework to express these things within the submodule, but I don't really have the bandwidth to take that on right now. Plus, who puts spaces in their filenames anymore? 😄

@gvwilson
Copy link

@gvwilson gvwilson commented Aug 17, 2013

This looks cool - will you be adding {% for %} and {% if %}, and if so, can they wrap cells, i.e., can I put:

+------------------------------+
| {% if instructor %} |
+------------------------------+
| ...a regular Python cell... |
+------------------------------+
| ...a Markdown cell... |
+------------------------------+
| {% endif %} |
+------------------------------+

and have it work as expected? Because I'd use that right away :-)

@jakevdp
Copy link
Contributor Author

@jakevdp jakevdp commented Aug 18, 2013

@gvwilson - that's a bit beyond the scope of this. The liquid tags are within the pelican markdown file, not within the notebook itself, so I'm not sure how that would work in this context.

@Carreau
Copy link

@Carreau Carreau commented Aug 18, 2013

@gvwilson this would go into nbconvert template/transformers, then in pelican you could do :

{% notebook /path/to/notebook.ipynb [ instructor | learner ] %}

@jakevdp
Copy link
Contributor Author

@jakevdp jakevdp commented Aug 20, 2013

Aside from several feature requests which I likely won't get to, I think this is ready to merge.
I've been using it with IPython 1.0 on my blog for about a week now, and all the bugs seem to be worked out.

There's certainly going to be work in the future to make it compatible with IPython 2.0, and perhaps add a better parsing framework than the raw regular expressions I'm currently using. Also, adding RST support would be nice, if someone wants to take that on.

None of those are blockers, though, and I'd like to finalize what we have.

@jakevdp
Copy link
Contributor Author

@jakevdp jakevdp commented Aug 26, 2013

A user noted that if sphinx is not installed, then the IPython version check was giving a misleading message. I fixed that.

@AlexMikhalev
Copy link

@AlexMikhalev AlexMikhalev commented Aug 27, 2013

What is the current status of this pull request?

@jakevdp
Copy link
Contributor Author

@jakevdp jakevdp commented Aug 27, 2013

Ready for merge

@justinmayer
Copy link
Member

@justinmayer justinmayer commented Aug 28, 2013

Thanks to Jake for all the hard work on this plugin, with due appreciation to others who assisted.

justinmayer added a commit that referenced this pull request Aug 28, 2013
@justinmayer justinmayer merged commit a20ca76 into getpelican:master Aug 28, 2013
@ocefpaf

This comment has been minimized.

Copy link

@ocefpaf ocefpaf commented on liquid_tags/notebook.py in 6cd4482 Sep 9, 2013

I believe that this comparison should be with a string ('1') instead of an integer (1).

This comment has been minimized.

Copy link
Owner Author

@jakevdp jakevdp replied Sep 9, 2013

Yes, this was fixed here: 1bda804

This comment has been minimized.

Copy link

@ocefpaf ocefpaf replied Sep 9, 2013

Thanks.

@jakevdp jakevdp deleted the jakevdp:liquid_tags branch Jun 7, 2018
@almet
Copy link
Member

@almet almet commented Jun 8, 2018

calfzhou pushed a commit to calfzhou/pelican-plugins that referenced this pull request Sep 28, 2019
Most users of the theme do not set any favicon. It results in useless
HTTP requests

Fix getpelican#21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

You can’t perform that action at this time.