Outline templates: use jinja file, instead of json (#62)

- extracted templates into separate files - use template outline file instead of json schema - improve front end logging - update documentation - version bump - added tests
chrisjsewell · Feb 1, 2019 · 5b4e82a · 5b4e82a
1 parent 608d936
commit 5b4e82a
Show file tree

Hide file tree

Showing 86 changed files with 1,288 additions and 1,361 deletions.
diff --git a/.vscode/settings.json b/.vscode/settings.json
@@ -14,6 +14,7 @@
     },
     "python.pythonPath": "/anaconda/envs/lsr/bin/python",
     "restructuredtext.confPath": "${workspaceFolder}/docs/source",
+    "restructuredtext.builtDocumentationPath": "${workspaceFolder}/docs/build/html",
     "restructuredtext.preview.scrollEditorWithPreview": false,
     "restructuredtext.preview.scrollPreviewWithEditor": false,
     "restructuredtext.updateOnTextChanged": "false",

diff --git a/MANIFEST.in b/MANIFEST.in
@@ -1,2 +1,3 @@
 recursive-include ipypublish *.json
+recursive-include ipypublish *.j2
 recursive-include ipypublish/tests/test_files *
diff --git a/bandit.yml b/bandit.yml
@@ -0,0 +1 @@
+skips: ['B101']  # TODO this should only skip in test files, see https://github.com/PyCQA/bandit/issues/346
diff --git a/docs/get_intersphinx_inv.py b/docs/get_intersphinx_inv.py
@@ -21,8 +21,8 @@ def warn(self, msg):
     from sphinx.ext import intersphinx
     import warnings
     # uri = 'http://jinja.pocoo.org/docs/dev/objects.inv'
-    # uri = "http://nbconvert.readthedocs.io/en/latest/objects.inv"
-    uri = "http://nbformat.readthedocs.io/en/latest/objects.inv"
+    uri = "http://nbconvert.readthedocs.io/en/latest/objects.inv"
+    # uri = "http://nbformat.readthedocs.io/en/latest/objects.inv"
 
     # Read inventory into a dictionary
     inv = fetch_inventory(uri)

diff --git a/docs/source/api/ipypublish.scripts.create_template.rst b/docs/source/api/ipypublish.scripts.create_template.rst
diff --git a/docs/source/api/ipypublish.scripts.rst b/docs/source/api/ipypublish.scripts.rst
@@ -6,7 +6,6 @@ Submodules
 
 .. toctree::
 
-   ipypublish.scripts.create_template
    ipypublish.scripts.ipynb_latex_setup
    ipypublish.scripts.nbmerge
    ipypublish.scripts.pdfexport

diff --git a/docs/source/api/ipypublish.templates.create_template.rst b/docs/source/api/ipypublish.templates.create_template.rst
@@ -0,0 +1,7 @@
+ipypublish\.templates\.create\_template module
+==============================================
+
+.. automodule:: ipypublish.templates.create_template
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/source/api/ipypublish.templates.outline_schemas.convert_format_str.rst b/docs/source/api/ipypublish.templates.outline_schemas.convert_format_str.rst
@@ -0,0 +1,7 @@
+ipypublish\.templates\.outline\_schemas\.convert\_format\_str module
+====================================================================
+
+.. automodule:: ipypublish.templates.outline_schemas.convert_format_str
+    :members:
+    :undoc-members:
+    :show-inheritance:
diff --git a/docs/source/api/ipypublish.templates.outline_schemas.rst b/docs/source/api/ipypublish.templates.outline_schemas.rst
@@ -1,6 +1,13 @@
 ipypublish\.templates\.outline\_schemas package
 ===============================================
 
+Submodules
+----------
+
+.. toctree::
+
+   ipypublish.templates.outline_schemas.convert_format_str
+
 Module contents
 ---------------
 

diff --git a/docs/source/api/ipypublish.templates.rst b/docs/source/api/ipypublish.templates.rst
@@ -9,6 +9,13 @@ Subpackages
     ipypublish.templates.outline_schemas
     ipypublish.templates.segments
 
+Submodules
+----------
+
+.. toctree::
+
+   ipypublish.templates.create_template
+
 Module contents
 ---------------
 

diff --git a/docs/source/custom_export_config.rst b/docs/source/custom_export_config.rst
@@ -73,7 +73,7 @@ The configuration file is a JSON file, with a validation schema given in
         "template": {
             "outline": {
                 "module": "ipypublish.templates.outline_schemas",
-                "file": "latex_tplx_schema.json"
+                "file": "latex_outline.latex.j2"
             },
             "segments": [
                 {
@@ -125,24 +125,30 @@ Template Construction
 ~~~~~~~~~~~~~~~~~~~~~
 
 In line 22, we define how to construct the `jinja`_ template.
-The ``outline`` key defines the path to an outline schema,
+The ``outline`` key defines the path to an outline template,
 such as in :ref:`outline_schema`.
-This file achieves two things; to define an outline of the `jinja`_ template
-structural blocks,
-with placeholders to be replaced by :py:func:`str.format`, and to
-provide a schema for segment files which are used to replace
-one or more of the placeholders.
+
+.. versionchanged:: 0.8.0
+
+    The outline file is now a jinja template, instead of a JSON file
+
+This template file can be a full jinja template file, extending
+an existing nbconvert template, but may optionally contain 'placeholders'
+(of the form ``@ipubreplace{below}{key_name}``)
+that can be replaced by injecting zero or more segments into them.
+The first option states whether segment injections are appended above or below
+previous injections, and the second option defines the key for that segment.
 
 This approach allows independent aspects of the document to be stored
 separately then pieced together in the desired manner. For example,
-the segment defined in :ref:`segment_config` defines only parts of the document
-which define how the bibliography is constructed.
+the segment file in :ref:`segment_config` defines only parts of the document
+which control how the bibliography is constructed.
 This could be removed or replaced by a custom export configuration.
 Similarly, input and output prompts can be added/removed in html documents.
 
-Segments are applied in the order they are defined,
-and the outline schema defines whether they are appended
-above or below existing content. For example, these segments:
+Segments are applied in the order they are defined, and appended
+above or below existing content, as defined by the placeholder.
+For example, these segments:
 
 .. code-block:: JSON
 
@@ -158,15 +164,27 @@ above or below existing content. For example, these segments:
         }
     ]
 
+applied to this template outline:
+
+.. code-block:: html+jinja
+
+    {% block markdowncell scoped %}
+    @ipubreplace{above}{notebook_input_markdown_pre}
+    @ipubreplace{below}{notebook_input_markdown}
+    @ipubreplace{below}{notebook_input_markdown_post}
+    {% endblock markdowncell %}
+
 will result in a template containing:
 
-.. code-block:: html
+.. code-block:: html+jinja
 
-   <div class='outer'>
-   <div class='inner'>
-     test
-   </div>
-   </div>
+    {% block markdowncell scoped %}
+    <div class='outer'>
+    <div class='inner'>
+        test
+    </div>
+    </div>
+    {% endblock markdowncell %}
 
 
 Segment configuration files also have an optional ``overwrite`` key, which

diff --git a/docs/source/examples.rst b/docs/source/examples.rst
@@ -0,0 +1,19 @@
+Examples
+--------
+
+For an example of the potential input/output:
+
+- :download:`Example.ipynb <../../example/notebooks/Example.ipynb>`
+- `Example.pdf <https://chrisjsewell.github.io/ipypublish/Example.view_pdf.html>`__
+- `Example.html <https://chrisjsewell.github.io/ipypublish/Example.html>`__
+- `Example.slides.html <https://chrisjsewell.github.io/ipypublish/Example.slides.html#/>`__
+
+.. todo:: replace external links with internal ones (but make sure content is correct)
+
+Or, for a practical example of the ipypublish capability, see these
+documents on Atomic 3D Visualisation:
+
+- `Notebook <https://github.com/chrisjsewell/chrisjsewell.github.io/blob/master/3d_atomic/3D%20Atomic%20Visualisation.ipynb>`__,
+- `PDF <https://chrisjsewell.github.io/3d_atomic/converted/3D%20Atomic%20Visualisation.view_pdf.html>`__,
+- `HTML <https://chrisjsewell.github.io/3d_atomic/converted/3D%20Atomic%20Visualisation.html>`__
+- `Reveal.JS slideshow <https://chrisjsewell.github.io/3d_atomic/converted/3D%20Atomic%20Visualisation.slides.html>`__.
diff --git a/docs/source/getting_started.rst b/docs/source/getting_started.rst
@@ -97,22 +97,4 @@ detailed log messages of the run are output to both the console and file
 use the ``--log-level debug`` and ``--pdf-debug`` flags. If there is
 still an error, please raise an issue on the `GitHub
 repository <https://github.com/chrisjsewell/ipypublish/issues>`__,
-including the run environment and the log file.
-
-Examples
---------
-
-See `Example.ipynb <example/notebooks/Example.ipynb>`__,
-`Example.pdf <https://chrisjsewell.github.io/ipypublish/Example.view_pdf.html>`__,
-`Example.html <https://chrisjsewell.github.io/ipypublish/Example.html>`__
-and
-`Example.slides.html <https://chrisjsewell.github.io/ipypublish/Example.slides.html#/>`__
-for an example of the potential input/output.
-
-Or, for a practical example of the ipypublish capability, see these
-documents on Atomic 3D Visualisation:
-`Notebook <https://github.com/chrisjsewell/chrisjsewell.github.io/blob/master/3d_atomic/3D%20Atomic%20Visualisation.ipynb>`__,
-`PDF <https://chrisjsewell.github.io/3d_atomic/converted/3D%20Atomic%20Visualisation.view_pdf.html>`__,
-`HTML <https://chrisjsewell.github.io/3d_atomic/converted/3D%20Atomic%20Visualisation.html>`__
-or `Reveal.JS
-slideshow <https://chrisjsewell.github.io/3d_atomic/converted/3D%20Atomic%20Visualisation.slides.html>`__.
+including the run environment and the log file.
diff --git a/docs/source/index.rst b/docs/source/index.rst
@@ -47,6 +47,7 @@ Citation
 
 Please cite |DOI| if using IPyPublish.
 
+
 Badges
 ======
 
@@ -57,6 +58,7 @@ Badges
    :caption: Contents:
 
    getting_started
+   examples
    notebook_conversion
    metadata_tags
    custom_export_config

diff --git a/docs/source/outline_schema.rst b/docs/source/outline_schema.rst
@@ -1,7 +1,7 @@
 .. _outline_schema:
 
-Template Outline Configuration Example
---------------------------------------
+Template Outline Example
+------------------------
 
-.. literalinclude:: ../../ipypublish/templates/outline_schemas/latex_tplx_schema.json
-  :language: JSON
+.. literalinclude:: ../../ipypublish/templates/outline_schemas/html_outline.html.j2
+  :language: html+jinja
diff --git a/ipypublish/__init__.py b/ipypublish/__init__.py
@@ -1 +1 @@
-__version__ = '0.7.1'
+__version__ = '0.8.0'