Integrate ipynb with pelican utilizing cell features to represent metadata and do other things in a pythonic way
Switch branches/tags
Nothing to show
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Type Name Latest commit message Commit time
Failed to load latest commit information.
test
.gitignore
README.md
__init__.py
math.py
preprocess.py
reader.py

README.md

ipynb2pelican Plugin

The ipynb2pelican plugin implements .ipynb file format support to pelican.

Features

  • KISS philosophy
    • Simply convert ipynb to html, no CSS is preserved
    • Code is simple to understand and to change
    • Using summary generation mechanism of pelican
  • Metadata Solution
    • MetaCell (Metadata Cell) is the first class citizen.
    • Liquid tag, additional metadata file is not considered in this project

MetaCell

All and Only Metadata should be stored at the first Cell of ipynb

Writing a MetaCell is as simple as writing metadata in markdown file.

# This is title
+ date: 2020-02-22
+ tags: hello, world

Hint: In jupyter notebook, press Esc+M will switch selected cell to markdown mode.

It is recommended (but not required) to organize the metadata items with Markdown bullets (+, - or *). These will be stripped during metadata extraction, but it keeps the metadata cell nicely readable in the notebook, like this:

This is title

  • date: 2020-02-22
  • tags: hello, world

Overview

The plugin is simple:

  • The CSS of jupyter will not be taken into outputs
  • The summary will be generated by Pelican

But it is still powerful and extensible:

  • Math Support
  • A Solution for metadata
  • Syntax hightlight by nbconvert
  • Several configurable preprocessors provided
    • Metadata Extraction
    • SubCell Selection
    • Ignore cells with #ignore tag
    • Empty Cell Removal
  • You can change preprocess.py and define your own preprocessors

Preprocessors

Thanks to the preprocessor feature of nbconvert, useful preprocessors are built in this project with on/off options. In your pelicanconf.py, you are able to set options to toggle preprocessors. You are encouraged to share your own preprocessors!

Don't worry about preprocessors, switch off all options will have only 3% gain on performance according to the test on my blog. So all of them are enabled by default.

Metadata Extraction

As we stated, All and Only Metadata should be stored at the first Cell of ipynb. If there is non-metadata content found, it will raise an exception. After the extraction of metadata, the MetaCell will be removed, as we have extracted all the information.

Summary cell IPYNB_SUMMARY_CELL

You could specify a jupyter cell as summary metadata of the article, which is useful in article lists. The summary cell functionality is based on metadata summarycell in metacell. Its value tells which cell to use or do not use any cell.

IPYNB_SUMMARY_CELL is False by default

  • If IPYNB_SUMMARY_CELL is True, then the default value of summarycell metadata is 1, i.e. the cell following the metadata cell
  • If IPYNB_SUMMARY_CELL is False, then the default value of summarycell metadata is 0, i.e. disabled

Use summary cell is super easy, just specify summarycell metadata into your metacell like this:

+ summarycell: value

If value is not specified, then it is taken as 1.

Remove Empty Cells IPYNB_REMOVE_EMPTY

Remove trivial cells without visible characters using regular expression \S

#ignore Tag IPYNB_IGNORE

You can include an #ignore comment at the beginning of a cell of the Jupyter notebook to ignore it, removing it from the post content.

Note it is more strict than #ignore tag in pelican-ipynb. The purpose is to prevent kicking normal contents out of post content.

SubCells Selection IPYNB_SUBCELLS

The Subcells preprocessor is executed after Metadata preprocessor (Th MetaCell it self will be removed by Metadata preprocessor), so zeroth cell is the first cell after MetaCell. The start and end should be written in the MetaCell like:

# This is title
+ date: 2020-02-22
+ tags: hello, world
+ subcells: [5, -1]

The value will be evaluated by start, end = ast.literal_eval(value). And then cells are sliced by cells[start:end].

Hint: If you want end to be infinity, use None

Options

Option Variable Default Meaning
IPYNB_REMOVE_EMPTY True Remove Empty Cells
IPYNB_IGNORE True Remove cells with #ignore tag at the beginning
IPYNB_SUBCELLS True Only preserve Subcells specified by subcells: [begin, end) metadata
IPYNB_SUMMARY_CELL False If summarycell is not in metadata, use first cell as summarycell
MATHJAX_CDN cdnjs The CDN of Mathjax to use

Installation and Configuration

Dependency

  • Both python2, python3 are supported
  • pelican
  • jupyter
  • ipython
  • nbconvert

It has been tested on

  • python 2.7/3.6
  • pelican 3.7.1
  • jupyter/ipython/nbconvert 4.1

Installation

Download this repo and rename folder to ipynb2pelican. Then put it into your plugins directory. A simpler way is by submodule this plugin:

git submodule add git@github.com:peijunz/ipynb2pelican.git pelican-plugins/ipynb2pelican

If your plugins are put into pelican-plugins directory, the file tree should looks like:

content/
pelicanconf.py
pelican-plugins/
└── ipynb2pelican
    ├── __init__.py
    ├── math.py
    ├── preprocess.py
    ├── reader.py
    └── README.md

Configuration

In the pelicanconf.py:

MARKUP = ('md', 'ipynb')                # Add 'ipynb'
PLUGIN_PATH = ['pelican-plugins']       # Ensure your plugin path is in it
PLUGINS = ['ipynb2pelican']             # Name of the plugin
IGNORE_FILES = ['.ipynb_checkpoints']   # Prevent parsing checkpoints files

If you are not using pelican-plugins, you should change it accordingly.