Skip to content
Permalink
main
Switch branches/tags
Go to file
 
 
Cannot retrieve contributors at this time
963 lines (751 sloc) 38.9 KB

Changes in nbconvert

6.1.0

This release is mostly a long list of bug fixes and capability additions. Thanks to the many contributors for helping Improve nbconvert!

The following 31 authors contributed 81 commits.

  • Adolph
  • Alessandro Finamore
  • Angus Hollands
  • Atsuo Ishimoto
  • Bo
  • David Brochart
  • Frédéric Collonval
  • Jeremy Howard
  • Jim Zwartveld
  • José Ignacio Romero
  • Joyce Er
  • joyceerhl
  • Kyle Cutler
  • Leah E. Cole
  • Leah Wasser
  • Matthew Seal
  • Michael Adolph
  • Mohammad Mostafa Farzan
  • Okky Mabruri
  • Pill-GZ
  • ptcane
  • Raniere Silva
  • Ryan Moe
  • Stefan Lang
  • Sylvain Corlay
  • Tobin Jones
  • txoof
  • Yuvi Panda

Significant Changes

Comprehensive notes

6.0.7

Primarly a release addressing template extensions issues reported since 6.0 launched.

Comprehensive notes

6.0.6

A range of bug fixes for webpdf exports

Comprehensive notes

  • Removed CSS preprocessor from default proprocessor list (fixes classic rendering) :ghpull:`1411`:
  • Fixed error when pickling TemplateExporter :ghpull:`1399`:
  • Support for fractional height html / webpdf exports :ghpull:`1413`:
  • Added short wait time for fonts and rendering in webpdf :ghpull:`1414`:
  • Updated template documentation
  • Minor fixes to the webpdf exporter :ghpull:`1419`:
  • Fixup use with a running event loop within webpdf :ghpull:`1420`:
  • Prevent overflow in input areas in lab template :ghpull:`1422`:

6.0.5

  • Revert networkidle2 change which caused custom cdn-fetched widgets in webpdf

6.0.4

Comprehensive notes

Fixing Problems

  • The webpdf exporters does not add pagebreaks anymore before reaching the maximum height allowed by Adobe :ghpull:`1402`:
  • Fixes some timeout issues with the webpdf exporter :ghpull:`1400`:

6.0.3

Execute preprocessor no longer add illegal execution counts to markdown cells :ghpull:`1396`:

6.0.2

A patch for a few minor issues raised out of the 6.0 release.

Comprehensive notes

Fixing Problems

  • Added windows work-around fix in CLI for async applications :ghpull:`1383`:
  • Fixed pathed template files to behave correctly for local relative paths without a dot :ghpull:`1381`:
  • ExecuteProcessor now properly has a preprocess_cell function to overwrite :ghpull:`1380`:

Testing, Docs, and Builds

  • Updated README and docs with guidance on how to get help with nbconvert :ghpull:`1377`:
  • Fixed documentation that was referencing template_path instead of template_paths :ghpull:`1374`:

6.0.1

A quick patch to fix an issue with get_exporter :ghpull:`1367`:

6.0

The following authors and reviewers contributed the changes for this release -- Thanks you all!

  • Ayaz Salikhov
  • bnables
  • Bo
  • David Brochart
  • David Cortés
  • Eric Wieser
  • Florian Rathgeber
  • Ian Allison
  • James Wilshaw
  • Jeremy Tuloup
  • Joel Ostblom
  • Jon Bannister
  • Jonas Drotleff
  • Josh Devlin
  • Karthikeyan Singaravelan
  • Kerwin.Sun
  • letmerecall
  • Luciano Resende
  • Lumír 'Frenzy' Balhar
  • Maarten A. Breddels
  • Maarten Breddels
  • Marcel Stimberg
  • Matthew Brett
  • Matthew Seal
  • Matthias Bussonnier
  • Matthias Geier
  • Miro Hrončok
  • Phil Austin
  • Praveen Batra
  • Ruben Di Battista
  • Ruby Werman
  • Sang-Yun Oh
  • Sergey Kizunov
  • Sundar
  • Sylvain Corlay
  • telamonian
  • Thomas Kluyver
  • Thomas Ytterdal
  • Tyler Makaro
  • Yu-Cheng (Henry) Huang

Significant Changes

Nbconvert 6.0 is a major release of nbconvert which includes many significant changes.

  • Python 2 support was dropped. Currently Python 3.6-3.8 is supported and tested by nbconvert. However, nbconvert 6.0 provides limited support for Python 3.6. nbconvert 6.1 will drop support for Python 3.6. Limited support means we will test and run CI on Python 3.6.12 or higher. Issues that are found only affecting Python 3.6 are not guaranteed to be fixed. We recommend all users of nbconvert use Python 3.7 and higher.
  • Unlike previous versions, nbconvert 6.0 relies on the nbclient package for the execute preprocessor, which allows for asynchronous kernel requests.
  • template_path has become template_paths. If referring to a 5.x style .tpl template use the full path with the template_file argument to the file. On the command line the pattern is --template-file=<path/to/file.tpl>.
  • Nbconvert 6.0 includes a new "webpdf" exporter, which renders notebooks in pdf format through a headless web browser, so that complex outputs such as HTML tables, or even widgets are rendered in the same way as with the HTML exporter and a web browser.
  • The default template applied when exporting to HTML now produces the same DOM structure as JupyterLab, and is styled using JupyterLab's CSS. The pygments theme in use mimics JupyterLab's codemirror mode with the same CSS variables, so that custom JupyterLab themes could be applied. The classic notebook styling can still be enabled with
jupyter nbconvert --to html --template classic
  • Nbconvert 6.0 includes a new system for creating custom templates, which can now be installed as packages. A custom "foobar" template is installed in Jupyter's data directory under nbconvert/templates and has the form of a directory containing all resources. Templates specify their base template as well as other configuration parameters in a conf.json at the root of the template directory.
  • The "slideshow" template now makes use of RevealJS version 4. It can now be used with the HTML exporter with
jupyter nbconvert --to html --template reveal

The --to slides exporter is still supported for convenience.

  • Inkscape 1.0 is now supported, which had some breaking changes that prevented 5.x versions of nbconvert from converting documents on some systems that updated.

Remaining changes

We merged 105 pull requests! Rather than enumerate all of them we'll link to the github page which contains the many smaller impact improvements.

The full list can be seen on GitHub

5.6.1

The following authors and reviewers contributed the changes for this release -- Thanks you all!

  • Charles Frye
  • Chris Holdgraf
  • Felipe Rodrigues
  • Gregor Sturm
  • Jim
  • Kerwin Sun
  • Ryan Beesley
  • Matthew Seal
  • Matthias Geier
  • thuy-van
  • Tyler Makaro

Significant Changes

RegExRemove applies to all cells

RegExRemove preprocessor now removes cells regardless of cell outputs. Before this only cells that had outputs were filtered.

Comprehensive notes

New Features

Fixing Problems

  • Update svg2pdf.py to search the PATH for inkscape :ghpull:`1115`:
  • Fix latex dependencies installation command for Ubuntu systems :ghpull:`1109`:

Testing, Docs, and Builds

5.6

The following 24 authors and reviewers contributed 224 commits -- Thank you all!

  • 00Kai0
  • Aidan Feldman
  • Alex Rudy
  • Alexander Kapshuna
  • Alexander Rudy
  • amniskin
  • Carol Willing
  • Dustin H
  • Hsiaoming Yang
  • imtsuki
  • Jessica B. Hamrick
  • KrokodileDandy
  • Kunal Marwaha
  • Matthew Seal
  • Matthias Geier
  • Miro Hrončok
  • M Pacer
  • Nils Japke
  • njapke
  • Sebastian Führ
  • Sylvain Corlay
  • Tyler Makaro
  • Valery M
  • Wayne Witzel

The full list of changes they made can be seen on GitHub

Significant Changes

Jupter Client Pin

The jupyter_client dependency is now pinned to >5.3.1. This is done to support the Parallel NBConvert below, and future versions may require interface changes from that version.

Parallel NBConvert

NBConvert --execute can now be run in parallel via threads, multiprocessing, or async patterns! This means you can now parallelize nbconvert via a bash loop, or a python concurrency pattern and it should be able to execute those notebooks in parallel.

Kernels have varying support for safe concurrent execution. The ipython kernel (ipykernel version 1.5.2 and higher) should be safe to run concurrently using Python 3. However, the Python 2 ipykernel does not always provide safe concurrent execution and sometimes fails with a socket bind exception. Unlike ipykernel which is maintained by the project, other community-maintained kernels may have varying support for concurrent execution, and these kernels were not tested heavily.

Issues for nbconvert can be viewed here: :ghpull:`1018`:, and :ghpull:`1017`:

Execute Loop Rewrite

This release completely rewrote the execution loop responsible for monitoring kernel messages until cell execution is completed. This removes an error where kernel messages could be dropped if too many were posted too quickly. Furthermore, the change means that messages are not buffered. Now, messages can be logged immediately rather than waiting for the cell to terminate.

See :ghpull:`994`: for exact code changes if you're curious.

Comprehensive notes

New Features

Fixing Problems

Testing, Docs, and Builds

5.5

The following 18 authors contributed 144 commits -- Thank you all!

  • Benjamin Ragan-Kelley
  • Clayton A Davis
  • DInne Bosman
  • Doug Blank
  • Henrique Silva
  • Jeff Hale
  • Lukasz Mitusinski
  • M Pacer
  • Maarten Breddels
  • Madhumitha N
  • Matthew Seal
  • Paul Gowder
  • Philipp A
  • Rick Lupton
  • Rüdiger Busche
  • Thomas Kluyver
  • Tyler Makaro
  • WrRan

The full list of changes they made can be seen on GitHub

Significant Changes

Deprecations

Python 3.4 support was dropped. Many of our upstream libraries stopped supporting 3.4 and it was found that serious bugs were being caught during testing against those libraries updating past 3.4.

See :ghpull:`979` for details.

IPyWidget Support

Now when a notebook executing contains Jupyter Widgets, the state of all the widgets can be stored in the notebook's metadata. This allows rendering of the live widgets on, for instance nbviewer, or when converting to html.

You can tell nbconvert to not store the state using the store_widget_state argument:

jupyter nbconvert --ExecutePreprocessor.store_widget_state=False --to notebook --execute mynotebook.ipynb

This widget rendering is not performed against a browser during execution, so only widget default states or states manipulated via user code will be calculated during execution. %%javascript cells will execute upon notebook rendering, enabling complex interactions to function as expected when viewed by a UI.

If you can't view widget results after execution, you may need to select :menuselection:`File --> Trust Notebook` in the menu.

See :ghpull:`779`, :ghpull:`900`, and :ghpull:`983` for details.

Execute Preprocessor Rework

Based on monkey patching required in papermill the run_cell code path in the ExecutePreprocessor was reworked to allow for accessing individual message parses without reimplementing the entire function. Now there is a process_message function which take a ZeroMQ message and applies all of its side-effect updates on the cell/notebook objects before returning the output it generated, if it generated any such output.

The change required a much more extensive test suite covering cell execution as test coverage on the various, sometimes wonky, code paths made improvements and reworks impossible to prove undamaging. Now changes to kernel message processing has much better coverage, so future additions or changes with specs over time will be easier to add.

See :ghpull:`905` and :ghpull:`982` for details

Out Of Memory Kernel Failure Catches

When running out of memory on a machine, if the kernel process was killed by the operating system it would result in a timeout error at best and hang indefinitely at worst. Now regardless of timeout configuration, if the underlying kernel process dies before emitting any messages to the effect an exception will be raised notifying the consumer of the lost kernel within a few seconds.

See :ghpull:`959`, :ghpull:`971`, and :ghpull:`998` for details

Latex / PDF Template Improvements

The latex template was long overdue for improvements. The default template had a rewrite which makes exports for latex and pdf look a lot better. Code cells in particular render much better with line breaks and styling the more closely matches notebook browser rendering. Thanks t-makaro for the efforts here!

See :ghpull:`992` for details

Comprehensive notes

New Features

Fixing Problems

Deprecations

Testing, Docs, and Builds

5.4.1

5.4.1 on Github

Thanks to the following 11 authors who contributed 57 commits.

  • Benjamin Ragan-Kelley
  • Carol Willing
  • Clayton A Davis
  • Daniel Rodriguez
  • M Pacer
  • Matthew Seal
  • Matthias Geier
  • Matthieu Parizy
  • Rüdiger Busche
  • Thomas Kluyver
  • Tyler Makaro

Comprehensive notes

New Features

Fixing Problems

Testing, Docs, and Builds

  • DOC: Add missing language specification to code-block :ghpull:`882`:

5.4

5.4 on Github

Significant Changes

Deprecations

Python 3.3 support was dropped. The version of python is no longer common and new versions have many fixes and interface improvements that warrant the change in support.

See :ghpull:`843` for implementation details.

Changes in how we handle metadata

There were a few new metadata fields which are now respected in nbconvert.

nb.metadata.authors metadata attribute will be respected in latex exports. Multiple authors will be added with , separation against their names.

nb.metadata.title will be respected ahead of nb.metadata.name for title assignment. This better matches with the notebook format.

nb.metadata.filename will override the default output_filename_template when extracting notebook resources in the ExtractOutputPreprocessor. The attribute is helpful for when you want to consistently fix to a particular output filename, especially when you need to set image filenames for your exports.

The raises-exception cell tag (nb.cells[].metadata.tags[raises-exception]) allows for cell exceptions to not halt execution. The tag is respected in the same way by nbval and other notebook interfaces. nb.metadata.allow_errors will apply this rule for all cells. This feature is toggleable with the force_raise_errors configuration option. Errors from executing the notebook can be allowed with a raises-exception tag on a single cell, or the allow_errors configurable option for all cells. An allowed error will be recorded in notebook output, and execution will continue. If an error occurs when it is not explicitly allowed, a 'CellExecutionError' will be raised. If force_raise_errors is True, CellExecutionError will be raised for any error that occurs while executing the notebook. This overrides both the allow_errors option and the raises-exception cell tags.

See :ghpull:`867`, :ghpull:`703`, :ghpull:`685`, :ghpull:`672`, and :ghpull:`684` for implementation changes.

Configurable kernel managers when executing notebooks

The kernel manager can now be optionally passed into the ExecutePreprocessor.preprocess and the executenb functions as the keyword argument km. This means that the kernel can be configured as desired before beginning preprocessing.

This is useful for executing in a context where the kernel has external dependencies that need to be set to non-default values. An example of this might be a Spark kernel where you wish to configure the Spark cluster location ahead of time without building a new kernel.

Overall the ExecutePreprocessor has been reworked to make it easier to use. Future releases will continue this trend to make this section of the code more inheritable and reusable by others. We encourage you read the source code for this version if you're interested in the detailed improvements.

See :ghpull:`852` for implementation changes.

Surfacing exporters in front-ends

Exporters are now exposed for front-ends to consume, including classic notebook. As an example, this means that latex exporter will be made available for latex 'text/latex' media type from the Download As interface.

See :ghpull:`759` and :ghpull:`864` for implementation changes.

Raw Templates

Template exporters can now be assigned raw templates as string attributes by setting the raw_template variable.

class AttrExporter(TemplateExporter):
    # If the class has a special template and you want it defined within the class
    raw_template = """{%- extends 'rst.tpl' -%}
{%- block in_prompt -%}
raw template
{%- endblock in_prompt -%}
    """
exporter_attr = AttrExporter()
output_attr, _ = exporter_attr.from_notebook_node(nb)
assert "raw template" in output_attr

See :ghpull:`675` for implementation changes.

New command line flags

The --no-input will hide input cells on export. This is great for notebooks which generate "reports" where you want the code that was executed to not appear by default in the extracts.

An alias for notebook was added to exporter commands. Now --to ipynb will behave as --to notebook does.

See :ghpull:`825` and :ghpull:`873` for implementation changes.

Comprehensive notes

New Features

Deprecations

  • Drop support for python 3.3 :ghpull:`843`
  • Default conversion method on the CLI was removed (--to html now required)

Fixing Problems

Testing, Docs, and Builds

5.3.1

5.3.1 on Github

  • MANIFEST.in updated to include LICENSE and scripts/ when creating sdist. :ghpull:`666`

5.3

5.3 on Github

Major features

Tag Based Element Filtering

For removing individual elements from notebooks, we need a way to signal to nbconvert that the elements should be removed. With this release, we introduce the use of tags for that purpose.

Tags are user-defined strings attached to cells or outputs. They are stored in cell or output metadata. For more on tags see the nbformat docs on cell metadata.

Usage:

  1. Apply tags to the elements that you want to remove.

For removing an entire cell, the cell input, or all cell outputs apply the tag to the cell.

For removing individual outputs, put the tag in the output metadata using a call like display(your_output_element, metadata={tags=[<your_tags_here>]}).

NB: Use different tags depending on whether you want to remove the entire cell, the input, all outputs, or individual outputs.

  1. Add the tags for removing the different kinds of elements to the following traitlets. Which kind of element you want to remove determines which traitlet you add the tags to.

The following traitlets remove elements of different kinds:

  • remove_cell_tags: removes cells
  • remove_input_tags: removes inputs
  • remove_all_outputs_tag: removes all outputs
  • remove_single_output_tag: removes individual outputs

Comprehensive notes

  • new: configurable browser in ServePostProcessor :ghpull:`618`
  • new: --clear-output command line flag to clear output in-place :ghpull:`619`
  • new: remove elements based on tags with TagRemovePreprocessor. :ghpull:`640`, :ghpull:`643`
  • new: CellExecutionError can now be imported from nbconvert.preprocessors :ghpull:`656`
  • new: slides now can enable scrolling and custom transitions :ghpull:`600`
  • docs: Release instructions for nbviewer-deploy
  • docs: improved instructions for handling errors using the ExecutePreprocessor :ghpull:`656`
  • tests: better height/width metadata testing for images in rst & html :ghpull:`601` :ghpull:`602`
  • tests: normalise base64 output data to avoid false positives :ghpull:`650`
  • tests: normalise ipython traceback messages to handle old and new style :ghpull:`631`
  • bug: mathjax obeys \\(\\) & \\[\\] (both nbconvert & pandoc) :ghpull:`609` :ghpull:`617`
  • bug: specify default templates using extensions :ghpull:`639`
  • bug: fix pandoc version number :ghpull:`638`
  • bug: require recent mistune version :ghpull:`630`
  • bug: catch errors from IPython execute_reply and error messages :ghpull:`642`
  • nose completely removed & dependency dropped :ghpull:`595` :ghpull:`660`
  • mathjax processing in mistune now only uses inline grammar :ghpull:`611`
  • removeRegex now enabled by default on all TemplateExporters, does not remove cells with outputs :ghpull:`616`
  • validate notebook after applying each preprocessor (allowing additional attributes) :ghpull:`645`
  • changed COPYING.md to LICENSE for more standard licensing that GitHub knows how to read :ghpull:`654`

5.2.1

5.2 on GitHub

Major features

In this release (along with the usual bugfixes and documentation improvements, which are legion) we have a few new major features that have been requested for a long time:

Global Content Filtering

You now have the ability to remove input or output from code cells, markdown cells and the input and output prompts. The easiest way to access all of these is by using traitlets like TemplateExporter.exclude_input = True (or, for example HTMLExporter.exclude_markdown = True if you wanted to make it specific to HTML output). On the command line if you just want to not have input or output prompts just use --no-prompt.

Execute notebooks from a function

You can now use the executenb function to execute notebooks as though you ran the execute preprocessor on the notebooks. It returns the standard notebook and resources options.

Remove cells based on regex pattern

This removes cells based on their matching a regex pattern (by default, empty cells). This is the RegexRemovePreprocessor.

Script exporter entrypoints for nonpython scripts

Now there is an entrypoint for having an exporter specific to the type of script that is being exported. While designed for use with the IRkernel in particular (with a script exporter focused on exporting R scripts) other non-python kernels that wish to have a language specific exporter can now surface that directly.

Comprehensive notes

Credits

This release has been larger than previous releases. In it 33 authors contributed a total of 546 commits.

Many thanks to the following individuals who contributed to this release (in alphabetical order):

  • Adam Chainz
  • Andreas Mueller
  • Bartosz T
  • Benjamin Ragan-Kelley
  • Carol Willing
  • Damián Avila
  • Elliot Marsden
  • Gao, Xiang
  • Jaeho Shin
  • Jan Schulz
  • Jeremy Kun
  • Jessica B. Hamrick
  • John B Nelson
  • juhasch
  • Livia Barazzetti
  • M Pacer
  • Matej Urbas
  • Matthias Bussonnier
  • Matthias Geier
  • Maximilian Albert
  • Michael Scott Cuthbert
  • Nicholas Bollweg
  • Paul Gowder
  • Paulo Villegas
  • Peter Parente
  • Philipp A
  • Scott Sanderson
  • Srinivas Reddy Thatiparthy
  • Sylvain Corlay
  • Thomas Kluyver
  • Till Hoffmann
  • Xiang Gao
  • YuviPanda

5.1.1

5.1.1 on GitHub

  • fix version numbering because of incomplete previous version number

5.1

5.1 on GitHub

5.0

5.0 on GitHub

  • Use :command:`xelatex` by default for latex export, improving unicode and font support.
  • Use entrypoints internally to access Exporters, allowing for packages to declare custom exporters more easily.
  • New ASCIIDoc Exporter.
  • New preprocessor for sanitised html output.
  • New general convert_pandoc filter to reduce the need to hard-code lists of filters in templates.
  • Use pytest, nose dependency to be removed.
  • Refactored Exporter code to avoid ambiguity and cyclic dependencies.
  • Update to traitlets 4.2 API.
  • Fixes for Unicode errors when showing execution errors on Python 2.
  • Default math font matches default Palatino body text font.
  • General documentation improvements. For example, testing, installation, custom exporters.
  • Improved link handling for LaTeX output
  • Refactored the automatic id generation.
  • New kernel_manager_class configuration option for allowing systems to be set up to resolve kernels in different ways.
  • Kernel errors now will be logged for debugging purposes when executing notebooks.

4.3

4.3 on GitHub

  • added live widget rendering for html output, nbviewer by extension

4.2

4.2 on GitHub

  • :ref:`Custom Exporters <external_exporters>` can be provided by external packages, and registered with nbconvert via setuptools entrypoints.
  • allow nbconvert reading from stdin with --stdin option (write into notebook basename)
  • Various ANSI-escape fixes and improvements
  • Various LaTeX/PDF export fixes
  • Various fixes and improvements for executing notebooks with --execute.

4.1

4.1 on GitHub

  • setuptools fixes for entrypoints on Windows
  • various fixes for exporters, including slides, latex, and PDF
  • fixes for exceptions met during execution
  • include markdown outputs in markdown/html exports

4.0

4.0 on GitHub