diff --git a/minted/Makefile b/minted/Makefile new file mode 100644 index 00000000..600a36cc --- /dev/null +++ b/minted/Makefile @@ -0,0 +1,65 @@ +.PHONY: all +all: sample_beamer.pdf sample_latex.pdf sample.html + +# NOTE: `pandoc_inputs` can have multiple filenames if you want to send `pandoc` +# more than one input file at once. In the commands for the targets that depend +# on `pandoc_inputs` you will see a pattern `$^ > $@`. It's less magic than it +# seems, but useful to point out if you have not seen these before. They are +# called "Automatic Variables", and more documentation can be found here: +# +# https://www.gnu.org/software/make/manual/html_node/Automatic-Variables.html +# +# So by depending on $(pandoc_inputs) and using $^ as the input files to +# `pandoc`, $^ will expand to all filenames in `pandoc_inputs` and the target +# will re-run when the timestamp of _any_ file listed in `pandoc_inputs` is +# updated. By redirecting the output to $@, it will send the `pandoc` output to +# the target name. In the examples below, $@ expands to either +# `sample_beamer.tex`, `sample_latex.tex`, or `sample.html` (depending on the +# target name). +# +# TL;DR: You should be able to copy-paste the commands below and just rename the +# target names to match whatever output filenames you want. +pandoc_inputs := sample.md + +# Sample beamer presentation. +sample_beamer.tex: $(pandoc_inputs) + pandoc -s -t beamer --lua-filter=minted.lua $^ > $@ + +sample_beamer.pdf: sample_beamer.tex + latexmk -pdf -shell-escape -jobname=sample_beamer sample_beamer + +# Sample latex document. +sample_latex.tex: $(pandoc_inputs) + pandoc -s -t latex --lua-filter=minted.lua $^ > $@ + +sample_latex.pdf: sample_latex.tex + latexmk -pdf -shell-escape -jobname=sample_latex sample_latex + +# Sample html5 document. +sample.html: $(pandoc_inputs) + pandoc -s -t html5 --lua-filter=minted.lua $^ > $@ + +# --- + +.PHONY: clean realclean +clean: + @# latexmk errors if no auxiliary files exist to cleanup. Using `|| true` + @# just makes it so that the subsequent commands will also execute. + latexmk -c sample_beamer >/dev/null 2>&1 || true + @# latexmk does not clean all beamer files + rm -f sample_beamer.{nav,snm,vrb} + rm -rf _minted-sample_beamer/ + latexmk -c sample_latex >/dev/null 2>&1 || true + rm -rf _minted-sample_latex/ + +realclean: clean + rm -f sample_beamer.{tex,pdf} + rm -f sample_latex.{tex,pdf} + rm -f sample.html + +.PHONY: test lint +lint: + flake8 --max-line-length=80 run_minted_tests.py background_color.py + +test: + @./run_minted_tests.py diff --git a/minted/README.md b/minted/README.md new file mode 100644 index 00000000..b2be0c0a --- /dev/null +++ b/minted/README.md @@ -0,0 +1,291 @@ +# minted + +This filter enables users to use the [`minted`][minted] package with the +`beamer` and `latex` writers. Users may attach any desired `minted` specific +styling / attributes to their code-blocks (or via document metadata). These +`minted` specific attributes will be _removed_ for any writers that are not +`beamer` or `latex`, since many of the `minted` options require using `latex` +specific syntax that can cause problems in other output formats. For example, +if the `fontsize=\footnotesize` attribute were applied to a code block, an +`html` export would include `data-fontsize="\footnotesize"`, which may produce +errors or more commonly be entirely meaningless for non-latex writers. + +The `minted` package will be used as a _replacement_ for the existing `pandoc` +inline code and code block elements. Behind the scenes, `minted` builds on top +of the `fancyvrb` latex package, using [pygments][pygments] to perform the +highlighting. The `minted` package contains _many_ options for customizing +output, users are encouraged to read / review section 5.3 of the +[minted documentation][minted_docs]. **This filter does not make any attempts +to validate arguments supplied to the `minted` package**. Invalid / conflicting +arguments are a usage error. + +**Contents** + +- [Setup](#setup) + - [LaTeX Preamble Configuration](#latex-preamble-configuration) + - [PDF Compilation](#pdf-compilation) +- [Minted Filter Settings](#minted-filter-settings) + - [Default Settings](#default-settings) + - [All Metadata Settings](#all-metadata-settings) + - [`no_default_autogobble`](#no_default_autogobble-boolean) + - [`no_mintinline`](#no_mintinline-boolean) + - [`default_block_language`](#default_block_language-string) + - [`default_inline_language`](#default_inline_language-string) + - [`block_attributes`](#block_attributes-list-of-strings) + - [`inline_attributes`](#inline_attributes-list-of-strings) +- [Important Usage Notes](#important-usage-notes) +- [Bonus](#bonus) + +# Setup + +## LaTeX Preamble Configuration + +Since this filter will emit `\mintline` commands for inline code, and +`\begin{minted} ... \end{minted}` environments for code blocks, you must ensure +that your document includes the `minted` package in the preamble of your +`beamer` or `latex` document. The filter cannot accomplish this for you. + +**Option 1** + +Use the `header-includes` feature of `pandoc` (`-H` / `--include-in-header`). +This will be injected into the preamble section of your `beamer` or `latex` +document. The bare minimum you need in this file is + +```latex +\usepackage{minted} +``` + +However, there are many other things you can set here (related or unrelated to +this filter), and this is a good opportunity to perform some global setup on the +`minted` package. Some examples: + +```latex +\usepackage{minted} + +% Set the `style=tango` attribute for all minted blocks. Can still be overriden +% per block (e.g., you want to change just one). Run `pygmentize -L` to see +% all available options. +\usemintedstyle{tango} + +% Depending on which pygments style you choose, comments and preprocessor +% directives may be italic. The `tango` style is one of these. This disables +% all italics in the `minted` environment. +\AtBeginEnvironment{minted}{\let\itshape\relax} + +% This disables italics for the `\mintinline` commands. +% Credit: https://tex.stackexchange.com/a/469702/113687 +\usepackage{xpatch} +\xpatchcmd{\mintinline}{\begingroup}{\begingroup\let\itshape\relax}{}{} +``` + +The `minted` package has many options, see the +[minted documentation][minted_docs] for more information. For example, see the +`bgcolor` option for the `minted` package. In this "header-include" file would +be an excellent location to `\definecolor`s that you want to use with `bgcolor`. + +**Option 1.5** + +You can also set `header-includes` in the metadata of your document. The above +example could be set as (noting the escaped backslashes): + +```yaml +colorlinks: true +header-includes: + # Include the minted package, set global style, define colors, etc. + - "\\usepackage{minted}" + - "\\usemintedstyle{tango}" + # Prevent italics in the `minted` environment. + - "\\AtBeginEnvironment{minted}{\\let\\itshape\\relax}" + # Prevent italics in the `\mintinline` command. + - "\\usepackage{xpatch}" + - "`\\xpatchcmd{\\mintinline}{\\begingroup}{\\begingroup\\let\\itshape\\relax}{}{}`{=latex}" +``` + +Note on the last line calling `\xpatchcmd`, we escape the backslashes and +additionally force `pandoc` to treat this as `latex` code by making it an inline +`latex` code element. See [pandoc issue 2139 (comment)][pandoc_issue_2139] for +more information. + +Formally, you may want to apply the ``-"`\\raw_tex`{=latex}"`` trick to all +metadata to indicate it is `latex` specific code. However, since `pandoc` +strips out any raw `latex` when converting to other writers, it isn't necessary. + +**Option 2** + +You can also create your own custom `beamer` or `latex` template to have much +finer control over what is / is not included in your document. You may obtain +a copy of the template that `pandoc` uses by default by running +`pandoc -D beamer` or `pandoc -D latex` depending on your document type. + +After you have modified the template to suit your needs (including at the very +least a `\usepackage{minted}`), specify your template file to `pandoc` using +the `--template ` command line argument. + +## PDF Compilation + +To compile a PDF, there are two things that the `minted` package requires be +available: an escaped shell to be able to run external commands (the +`-shell-escape` command line flag), and the ability to create and later read +auxiliary files (`minted` runs `pygmentize` for the highlighting). + +At the time of writing this, only one of these is accessible using `pandoc` +directly. One may pass `--pdf-engine-opt=-shell-escape` to forward the +`-shell-escape` flag to the latex engine being used. Unfortunately, though, +the second component (related to temporary files being created) is not supported +by `pandoc`. See [pandoc issue 4271][pandoc_issue_4271]. + +**However**, in reality this is an minor issue that can easily be worked around. +Instead of generating `md => pdf`, you just use `pandoc` to generate `md => tex` +and then compile `tex => pdf` yourself. See the [sample Makefile](Makefile) for +examples of how to execute both stages. **Furthermore**, you will notice a +significant advantage of managing the `pdf` compilation yourself: the generated +`minted` files are cached and unless you `make clean` (or remove them manually), +unchanged code listings will be reused. That is, you will have faster +compilation times :slightly_smiling_face: + +# Minted Filter Settings + +Direct control over the settings of this filter are performed by setting +sub-keys of a `minted` metadata key for your document. + +## Default Settings + +By default, this filter + +1. Transforms all inline `Code` elements to `\mintinline`. This can be disabled + globally by setting `no_mintinline: true`. + +2. Transforms all `CodeBlock` elements to `\begin{minted} ... \end{minted}` raw + latex code. This cannot be disabled. + +3. Both (1) and (2) default to the `"text"` pygments lexer, meaning that inline + code or code blocks without a specific code class applied will receive no + syntax highlighting. This can be changed globally by setting + `default_block_language: "lexer"` or `default_inline_language: "lexer"`. + +4. All `CodeBlock` elements have the `autogobble` attribute applied to them, + which informs `minted` to trim all common preceding whitespace. This can be + disabled globally by setting `no_default_autogobble: true`. However, doing + this is **strongly discouraged**. Consider a code block nested underneath + a list item. Pandoc will (correctly) generate indented code, meaning you + will need to manually inform `minted` to `gobble=indent` where `indent` is + the number of spaces to trim. Note that `pandoc` may not reproduce the same + indentation level of the original document. + +## All Metadata Settings + +Each of the following are nested under the `minted` metadata key. + +### `no_default_autogobble` (boolean) + +By default this filter will always use `autogobble` with minted, which will +automatically trim common preceding whitespace. This is important because +code blocks nested under a list or other block elements _will_ have common +preceding whitespace that you _will_ want trimmed. + +### `no_mintinline` (boolean) + +Globally prevent this filter from emitting `\mintinline` calls for inline +Code elements, emitting `\texttt` instead. Possibly useful in saving +compile time for large documents that do not seek to have syntax +highlighting on inline code elements. + +### `default_block_language` (string) + +The default pygments lexer class to use for code blocks. By default this +is `"text"`, meaning no syntax highlighting. This is a fallback value, code +blocks that explicitly specify a lexer will not use it. + +### `default_inline_language` (string) + +Same as `default_block_language`, only for inline code (typed in single +backticks). The default is also `"text"`, and changing is discouraged. + +### `block_attributes` (list of strings) + +Any default attributes to apply to _all_ code blocks. These may be +overriden on a per-code-block basis. See section 5.3 of the +[minted documentation][minted_docs] for available options. + +### `inline_attributes` (list of strings) + +Any default attributes to apply to _all_ inline code. These may be +overriden on a per-code basis. See section 5.3 of the +[minted documentation][minted_docs] for available options. + +[minted_docs]: http://mirrors.ctan.org/macros/latex/contrib/minted/minted.pdf +[minted]: https://ctan.org/pkg/minted?lang=en +[pygments]: http://pygments.org/ +[pandoc_issue_2139]: https://github.com/jgm/pandoc/issues/2139#issuecomment-310522113 +[pandoc_issue_4271]: https://github.com/jgm/pandoc/issues/4721 + +# Important Usage Notes + +Refer to the [`sample.md`](sample.md) file for some live examples of how to use +this filter. If you execute `make` in this directory, `sample_beamer.pdf`, +`sample_latex.pdf`, and `sample.html` will all be generated to demonstrate the +filter in action. + +`pandoc` allows you to specify additional attributes on either the closing +backtick of an inline code element, or after the third backtick of a fenced +code block. This is done using `{curly braces}`, an example: + +```md +`#include `{.cpp .showspaces style=bw} +``` + +or + + ```{.cpp .showspaces style=bw} + #include + ``` + +In order, these are + +- `.cpp`: specify the language lexer class. +- `.showspaces`: a `minted` boolean attribute. +- `style=bw`: a `minted` attribute that takes an argument (`bw` is a pygments + style, black-white, just an example). + +There are two rules that must not be violated: + +1. Any time you want to supply extra arguments to `minted` to a specific inline + code or code block element, **the lexer class must always be first, and + always be present**. + + This is a limitation of the implementation of this filter. + +2. Observe the difference between specifying boolean attributes vs attributes + that take an argument. Boolean `minted` attributes **must** have a leading + `.`, and `minted` attributes that take an argument **may not** have a leading + `.`. + + - **Yes**: `{.cpp .showspaces}`, **No**: `{.cpp showspaces}` + - **Yes**: `{.cpp style=bw}`, **No**: `{.cpp .style=bw}` + + If you violate this, then `pandoc` will likely not produce an actual inline + `Code` or `CodeBlock` element, but instead something else (undefined). + +# Bonus + +Included here is a simple python script to help you get the right color +definitions for `bgcolor` with minted. Just run +[`background_color.py`](background_color.py) with a single argument that is the +name of the pygments style you want the `latex` background color definition for: + +```console +$ ./background_color.py monokai +Options for monokai (choose *one*): + + (*) \definecolor{monokai_bg}{HTML}{272822} + (*) \definecolor{monokai_bg}{RGB}{39,40,34} + (*) \definecolor{monokai_bg}{rgb}{0.1529,0.1569,0.1333} + |--------/ + | + +--> You can rename this too :) +``` + +See the contents of [`sample.md`](sample.md) (click on "View Raw" to see the +comments in the metadata section). Notably, in order to use `\definecolor` you +should make sure that the `xcolor` package is actually included. Comments in +the file explain the options. diff --git a/minted/background_color.py b/minted/background_color.py new file mode 100755 index 00000000..e830db49 --- /dev/null +++ b/minted/background_color.py @@ -0,0 +1,76 @@ +#!/usr/bin/env python + +r""" +A simple script to print out the RGB ``\definecolor`` command for the background +color of a specified pygments style name. +""" + +import sys +try: + from pygments.styles import get_style_by_name +except ImportError as ie: + sys.stderr.write("Please install the Pygments package:\n{0}\n".format(ie)) + sys.exit(1) + + +if __name__ == "__main__": + # Make sure we have a style name provided. + if len(sys.argv) != 2: + sys.stderr.write("Usage: {0} \n\n".format(sys.argv[0])) + sys.stderr.write(" Tip: run `pygmentize -L` to see style names.\n") + sys.exit(1) + + # Grab the style object, error out if invalid name. + style_name = sys.argv[1] + try: + style = get_style_by_name(style_name) + except Exception as e: + sys.stderr.write("Unable to find {0}:\n{1}\n".format(style_name, e)) + sys.exit(1) + + # Convert the hexadecimal string into rgb. + background_hex = style.background_color.replace("#", "") + if len(background_hex) != 6: + sys.stderr.write("Unknown hex color: {0}\n".format(background_hex)) + sys.exit(1) + + try: + r = int(background_hex[0:2], 16) + g = int(background_hex[2:4], 16) + b = int(background_hex[4:6], 16) + except Exception as e: + sys.stderr.write("Unable to convert to integers:\n{0}\n".format(e)) + sys.exit(1) + + # Build out the various options for \definecolor + # All should be equivalent, but users may have a preference of one format + # over another :p + tex_color_name = "{0}_bg".format(style_name) + def_HTML = r"\definecolor{{{0}}}{{HTML}}{{{1}}}".format( + tex_color_name, background_hex.upper() + ) + def_RGB = r"\definecolor{{{0}}}{{RGB}}{{{1}}}".format( + tex_color_name, "{0},{1},{2}".format(r, g, b) + ) + def_rgb = r"\definecolor{{{0}}}{{rgb}}{{{1}}}".format( + tex_color_name, + ",".join(["{0:.4}".format(float(c) / 255.0) for c in [r, g, b]]) + ) + + # Enumerate the options + print("Options for {0} (choose *one*):\n".format(style_name)) + print(" (*) {0}".format(def_HTML)) + print(" (*) {0}".format(def_RGB)) + print(" (*) {0}".format(def_rgb)) + + # Make sure they know that `{style_name}_bg` can be changed to whatever + # they want to be using in their document. + notice = "{0}|{1}/".format( + len(r" (*) \definecolor{") * " ", + (len(tex_color_name) - 2) * "-" + ) + vline = notice[0:notice.find("|")+1] + can_change = vline.replace("|", "+--> You can rename this too :)") + print(notice) + print(vline) + print(can_change) diff --git a/minted/minted.lua b/minted/minted.lua new file mode 100644 index 00000000..b2baaa4f --- /dev/null +++ b/minted/minted.lua @@ -0,0 +1,421 @@ +--[[ +minted -- enable the minted environment for code listings in beamer and latex. + +MIT License + +Copyright (c) 2019 Stephen McDowell + +Permission is hereby granted, free of charge, to any person obtaining a copy +of this software and associated documentation files (the "Software"), to deal +in the Software without restriction, including without limitation the rights +to use, copy, modify, merge, publish, distribute, sublicense, and/or sell +copies of the Software, and to permit persons to whom the Software is +furnished to do so, subject to the following conditions: + +The above copyright notice and this permission notice shall be included in all +copies or substantial portions of the Software. + +THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR +IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, +FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE +AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER +LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, +OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE +SOFTWARE. +]] + +-------------------------------------------------------------------------------- +-- Quick documentation. See full documentation here: -- +-- https://github.com/pandoc/lua-filters/blob/master/minted -- +-------------------------------------------------------------------------------- +--[[ +Brief overview of metadata keys that you can use in your document: + +minted: + no_default_autogobble: , *DISCOURAGED* + no_mintinline: + default_block_language: + default_inline_language: + block_attributes: + - attr_1 + - attr_2 + - ... + inline_attributes: + - attr_1 + - attr_2 + - ... + +In words, underneath the `minted` metadata key, you have the following options: + +### `no_default_autogobble` (boolean) + +By default this filter will always use `autogobble` with minted, which will +automatically trim common preceding whitespace. This is important because +code blocks nested under a list or other block elements _will_ have common +preceding whitespace that you _will_ want trimmed. + +### `no_mintinline` (boolean) + +Globally prevent this filter from emitting `\mintinline` calls for inline +Code elements, emitting `\texttt` instead. Possibly useful in saving +compile time for large documents that do not seek to have syntax +highlighting on inline code elements. + +### `default_block_language` (string) + +The default pygments lexer class to use for code blocks. By default this +is `"text"`, meaning no syntax highlighting. This is a fallback value, code +blocks that explicitly specify a lexer will not use it. + +### `default_inline_language` (string) + +Same as `default_block_language`, only for inline code (typed in single +backticks). The default is also `"text"`, and changing is discouraged. + +### `block_attributes` (list of strings) + +Any default attributes to apply to _all_ code blocks. These may be +overriden on a per-code-block basis. See section 5.3 of the +[minted documentation][minted_docs] for available options. + +### `inline_attributes` (list of strings) + +Any default attributes to apply to _all_ inline code. These may be +overriden on a per-code basis. See section 5.3 of the +[minted documentation][minted_docs] for available options. + +[minted_docs]: http://mirrors.ctan.org/macros/latex/contrib/minted/minted.pdf +]] + +local List = require('pandoc.List') + +-------------------------------------------------------------------------------- +-- Potential metadata elements to override. -- +-------------------------------------------------------------------------------- +local minted_no_mintinline = false +local minted_default_block_language = "text" +local minted_default_inline_language = "text" +local minted_block_attributes = {} +local minted_inline_attributes = {} + +-------------------------------------------------------------------------------- +-- Constants used to differentiate Code and CodeBlock elements. -- +-------------------------------------------------------------------------------- +local MintedInline = 0 +local MintedBlock = 1 + +-------------------------------------------------------------------------------- +-- Utility functions. -- +-------------------------------------------------------------------------------- +-- Return the string lexer class to be used with minted. `elem` should be +-- either a Code or CodeBlock element (whose `classes` list will be inspected +-- first). `kind` is assumed to be either `MintedInline` or `MintedBlock` in +-- order to choose the appropriate fallback lexer when unspecified. +local function minted_language(elem, kind) + -- If the code [block] attached classes, we assume the first one is the + -- lexer class to use. + if #elem.classes > 0 then + return elem.classes[1] + end + -- Allow user-level metadata to override the inline language. + if kind == MintedInline then + return minted_default_inline_language + end + -- Allow user-level metadata to override the block language. + if kind == MintedBlock then + return minted_default_block_language + end + + -- Failsafe, should not hit here unless function called incorrectly. + return "text" +end + +-- Returns a boolean specifying whether or not the specified string `cls` is an +-- option that is supported by the minted package. +local function is_minted_class(cls) + -- Section 5.3 Available Options of Minted documentation. Note that many of + -- these do not apply to \mintinline (inline Code). Users are responsible + -- for supplying valid arguments to minted. For example, specifying + -- `autogobble` and `gobble` at the same time is a usage error. + -- + -- http://mirrors.ctan.org/macros/latex/contrib/minted/minted.pdf + local all_minted_options = List:new{ + "autogobble", "baselinestretch", "beameroverlays", "breakafter", + "breakaftergroup", "breakaftersymbolpre", "breakaftersymbolpost", + "breakanywhere", "breakanywheresymbolpre", "breakanywheresymbolpost", + "breakautoindent", "breakbefore", "breakbeforegroup", + "breakbeforesymbolpre", "breakbeforesymbolpost", "breakbytoken", + "breakbytokenanywhere", "breakindent", "breakindentnchars", "breaklines", + "breaksymbol", "breaksymbolleft", "breaksymbolright", "breaksymbolindent", + "breaksymbolindentnchars", "breaksymbolindentleft", + "breaksymbolindentleftnchars", "breaksymbolindentright", + "breaksymbolindentrightnchars", "breaksymbolsep", "breaksymbolsepnchars", + "breaksymbolsepleft", "breaksymbolsepleftnchars", "breaksymbolsepright", + "breaksymbolseprightnchars", "bgcolor", "codetagify", "curlyquotes", + "encoding", "escapeinside", "firstline", "firstnumber", "fontfamily", + "fontseries", "fontsize", "fontshape", "formatcom", "frame", "framerule", + "framesep", "funcnamehighlighting", "gobble", "highlightcolor", + "highlightlines", "keywordcase", "label", "labelposition", "lastline", + "linenos", "numberfirstline", "numbers", "mathescape", "numberblanklines", + "numbersep", "obeytabs", "outencoding", "python3", "resetmargins", + "rulecolor", "samepage", "showspaces", "showtabs", "space", "spacecolor", + "startinline", "style", "stepnumber", "stepnumberfromfirst", + "stepnumberoffsetvalues", "stripall", "stripnl", "tab", "tabcolor", + "tabsize", "texcl", "texcomments", "xleftmargin", "xrightmargin" + } + return all_minted_options:includes(cls, 0) +end + +-- Return a string for the minted attributes `\begin{minted}[attributes]` or +-- `\mintinline[attributes]`. Attributes are acquired by inspecting the +-- specified element's `classes` and `attr` fields. Any global attributes +-- provided in the document metadata will be included _only_ if they do not +-- override the element-level attributes. +-- +-- `elem` should either be a Code or CodeBlock element, and `kind` is assumed to +-- be either `MintedInline` or `MintedBlock`. The `kind` determines which +-- global default attribute list to use. +local function minted_attributes(elem, kind) + -- The full listing of attributes that will be joined and returned. + local minted_attributes = {} + + -- Book-keeping, track xxx=yyy keys `xxx` that have been added to + -- `minted_attributes` to make checking optional global defaults via the + -- `block_attributes` or `inline_attributes` easier. + local minted_keys = {} + + -- Boolean style options for minted (e.g., ```{.bash .autogobble}) will appear + -- in the list of classes. + for _, cls in ipairs(elem.classes) do + if is_minted_class(cls) then + table.insert(minted_attributes, cls) + table.insert(minted_keys, cls) + end + end + + -- Value options using key=value (e.g., ```{.bash fontsize=\scriptsize}) show + -- up in the list of attributes. + for _, attr in ipairs(elem.attributes) do + cls, value = attr[1], attr[2] + if is_minted_class(cls) then + table.insert(minted_attributes, cls .. "=" .. value) + table.insert(minted_keys, cls) + end + end + + -- Add any global defaults _only_ if they do not conflict. Note that conflict + -- is only in the literal sense. If a user has `autogobble` and `gobble=2` + -- specified, these do conflict in the minted sense, but this filter makes no + -- checks on validity ;) + local global_defaults = nil + if kind == MintedInline then + global_defaults = minted_inline_attributes + elseif kind == MintedBlock then + global_defaults = minted_block_attributes + end + for _, global_attr in ipairs(global_defaults) do + -- Either use the index of `=` minus one, or -1 if no `=` present. Fallback + -- on -1 means that the substring is the original string. + local end_idx = (string.find(global_attr, "=") or 0) - 1 + local global_key = string.sub(global_attr, 1, end_idx) + local can_insert_global = true + for _, existing_key in ipairs(minted_keys) do + if existing_key == global_key then + can_insert_global = false + break + end + end + + if can_insert_global then + table.insert(minted_attributes, global_attr) + end + end + + -- Return a comma delimited string for specifying the attributes to minted. + return table.concat(minted_attributes, ",") +end + +-- Return the specified `elem` with any minted data removed from the `classes` +-- and `attr`. Otherwise writers such as the HTML writer might produce invalid +-- code since latex makes heavy use of the \backslash. +local function remove_minted_attibutes(elem) + -- Remove any minted items from the classes. + classes = {} + for _, cls in ipairs(elem.classes) do + if not is_minted_class(cls) and cls ~= "no_minted" then + table.insert(classes, cls) + end + end + elem.classes = classes + + -- Remove any minted items from the attributes. + extra_attrs = {} + for _, attr in ipairs(elem.attributes) do + cls, value = attr[1], attr[2] + if not is_minted_class(cls) then + table.insert(extra_attrs, {cls, value}) + end + end + elem.attributes = extra_attrs + + -- Return the (potentially modified) element for pandoc to take over. + return elem +end + +-------------------------------------------------------------------------------- +-- Pandoc overrides. -- +-------------------------------------------------------------------------------- +-- Override the pandoc Meta function so that we can parse the metadata for the +-- document and store the necessary variables locally to use in other functions +-- such as Code and CodeBlock (helper methods). +function Meta(m) + -- Grab the `minted` metadata, quit early if not present. + local minted = m["minted"] + local found_autogobble = false + local always_autogobble = true + if minted ~= nil then + -- Parse and set the global bypass to turn off all \mintinline calls. + local no_mintinline = minted["no_mintinline"] + if no_mintinline ~= nil then + minted_no_mintinline = no_mintinline + end + + -- Parse and set the default block language. + local default_block_language = minted.default_block_language + and pandoc.utils.stringify(minted.default_block_language) + if default_block_language ~= nil then + minted_default_block_language = default_block_language + end + + -- Parse and set the default inline language. + local default_inline_language = minted.default_inline_language + and pandoc.utils.stringify(minted.default_inline_language) + if default_inline_language ~= nil then + minted_default_inline_language = default_inline_language + end + + -- Parse the global default minted attributes to use on every block. + local block_attributes = minted["block_attributes"] + if block_attributes ~= nil then + for _, attr in ipairs(block_attributes) do + if attr == "autogobble" then + found_autogobble = true + end + table.insert(minted_block_attributes, attr[1].text) + end + end + + -- Allow users to turn off autogobble for blocks, but really they should not + -- ever seek to do this (indented code blocks under list for example). + local no_default_autogobble = minted["no_default_autogobble"] + if no_default_autogobble ~= nil then + always_autogobble = not no_default_autogobble + end + + -- Parse the global default minted attributes to use on ever inline. + local inline_attributes = minted["inline_attributes"] + if inline_attributes ~= nil then + for _, attr in ipairs(inline_attributes) do + table.insert(minted_inline_attributes, attr[1].text) + end + end + end + + -- Make sure autogobble is turned on by default if no `minted` meta key is + -- provided for the document. + if always_autogobble and not found_autogobble then + table.insert(minted_block_attributes, "autogobble") + end + + -- Return the metadata to pandoc (unchanged). + return m +end + +-- Override inline code elements to use \mintinline for beamer / latex writers. +-- Other writers have all minted attributes removed. +function Code(elem) + if FORMAT == "beamer" or FORMAT == "latex" then + -- Allow a bypass to turn off \mintinline via adding .no_minted class. + local found_no_minted_class = false + for _, cls in ipairs(elem.classes) do + if cls == "no_minted" then + found_no_minted_class = true + break + end + end + + -- Check for local or global bypass to turn off \mintinline + if minted_no_mintinline or found_no_minted_class then + return pandoc.RawInline("latex", string.format("\\texttt{%s}", elem.text)) + end + + local language = minted_language(elem, MintedInline) + local attributes = minted_attributes(elem, MintedInline) + local raw_minted = string.format( + "\\mintinline[%s]{%s}{%s}", + attributes, + language, + elem.text + ) + -- NOTE: prior to pandoc commit 24a0d61, `beamer` cannot be used as the + -- RawBlock format. Using `latex` should not cause any problems. + return pandoc.RawInline("latex", raw_minted) + else + return remove_minted_attibutes(elem) + end +end + +-- Override code blocks to use \begin{minted}...\end{minted} for beamer / latex +-- writers. Other writers have all minted attributes removed. +function CodeBlock(block) + if FORMAT == "beamer" or FORMAT == "latex" then + local language = minted_language(block, MintedBlock) + local attributes = minted_attributes(block, MintedBlock) + local raw_minted = string.format( + "\\begin{minted}[%s]{%s}\n%s\n\\end{minted}", + attributes, + language, + block.text + ) + -- NOTE: prior to pandoc commit 24a0d61, `beamer` cannot be used as the + -- RawBlock format. Using `latex` should not cause any problems. + return pandoc.RawBlock("latex", raw_minted) + else + return remove_minted_attibutes(block) + end +end + +-- Override headers to make all beamer frames fragile, since any minted +-- environments or \mintinline invocations will halt compilation if the frame +-- is not marked as fragile. +function Header(elem) + if FORMAT == 'beamer' then + -- Check first that 'fragile' is not already present. + local has_fragile = false + for _, val in ipairs(elem.classes) do + if val == 'fragile' then + has_fragile = true + break + end + end + + -- If not found, add fragile to the list of classes. + if not has_fragile then + table.insert(elem.classes, 'fragile') + end + + -- NOTE: pass the remaining work to pandoc, noting that 2.5 and below + -- may duplicate the 'fragile' specifier. Duplicated fragile does *not* + -- cause compile errors. + return elem + end +end + +-- NOTE: order of return matters, Meta needs to be first otherwise the metadata +-- from the document will not be loaded _first_. +return { + {Meta = Meta}, + {Code = Code}, + {CodeBlock = CodeBlock}, + {Header = Header} +} diff --git a/minted/run_minted_tests.py b/minted/run_minted_tests.py new file mode 100755 index 00000000..41fe4027 --- /dev/null +++ b/minted/run_minted_tests.py @@ -0,0 +1,454 @@ +#!/usr/bin/env python + +""" +Unit tests for the pandoc minted.lua filter. +""" + +# Lint this file with: flake8 --max-line-length=80 +import os +import subprocess +import sys +import textwrap + +code_block = textwrap.dedent(''' + ## A Code Block + + ```{.cpp} + auto mult = [](T const & x, U const & y) { + return x * y; + }; + ``` +''') +""" +The base CodeBlock code. {.cpp} is used as a replacement marker in most tests! +""" + +inline_code = textwrap.dedent(''' + ## Inline Code + + `#include `{.cpp} +''') +""" +The base Code code. {.cpp} is used as a replacement marker in most tests! +""" + + +def run_pandoc(pandoc_args, stdin): + """Run pandoc with the specified arguments, returning the output.""" + # The input / output should be small enough for these tests that buffer + # overflows should not happen. + pandoc_proc = subprocess.Popen( + ["pandoc"] + pandoc_args, + stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE + ) + + # Python 3.x and later require communicating with bytes. + if sys.version_info[0] >= 3: + stdin = bytes(stdin, "utf-8") + + stdout, stderr = pandoc_proc.communicate(input=stdin) + if pandoc_proc.returncode != 0: + sys.stderr.write("Non-zero exit code of {ret} from pandoc!\n".format( + ret=pandoc_proc.returncode + )) + sys.stderr.write("pandoc stderr: {stderr}".format( + stderr=stderr.decode("utf-8") + )) + sys.exit(1) + + return stdout.decode("utf-8") + + +def fail_test(test_name, messages, ansi_color_code="31"): + """ + Print failure message and ``sys.exit(1)``. + + ``test_name`` (str) + The name of the test (to make finding in code easier). + + ``messages`` (list of str -- or -- str) + A single string, or list of strings, to print out to ``stderr`` that + explain the reason for the test failure. + + ``ansi_color_code`` (str) + A an ANSI color code to use to colorize the failure message :) Default + is ``"31"``, which is red. + """ + sys.stderr.write( + "\033[0;{ansi_color_code}mTest {test_name} FAILED\033[0m\n".format( + ansi_color_code=ansi_color_code, test_name=test_name + ) + ) + if isinstance(messages, list): + for m in messages: + sys.stderr.write("--> {m}\n".format(m=m)) + else: + sys.stderr.write("--> {messages}\n".format(messages=messages)) + sys.exit(1) + + +def ensure_fragile(test_name, pandoc_output): + r""" + Ensure that every \begin{frame} has (at least one) fragile. + + ``test_name`` (str) + The name of the test (forwards to ``fail_test``). + + ``pandoc_output`` (str) + The pandoc output for the test case. + """ + for line in pandoc_output.splitlines(): + if r"\begin{frame}" in line: + if "fragile" not in line: + fail_test( + test_name, + r"\begin{frame} without 'fragile': {line}".format(line=line) + ) + + +def ensure_present(test_name, string, pandoc_output): + """ + Assert that ``string`` is found in ``pandoc_output``. + + ``test_name`` (str) + The name of the test (forwards to ``fail_test``). + + ``string`` (str) + The string to check verbatim ``string in pandoc_output``. + + ``pandoc_output`` (str) + The pandoc output for the test case. + """ + if string not in pandoc_output: + fail_test( + test_name, + "The requested string '{string}' was not found in:\n{pout}".format( + string=string, pout=pandoc_output + ) + ) + + +def ensure_not_present(test_name, string, pandoc_output): + """ + Assert that ``string`` is **not** found in ``pandoc_output``. + + ``test_name`` (str) + The name of the test (forwards to ``fail_test``). + + ``string`` (str) + The string to check verbatim ``string not in pandoc_output``. + + ``pandoc_output`` (str) + The pandoc output for the test case. + """ + if string in pandoc_output: + fail_test( + test_name, + "The forbidden string '{string}' was found in:\n{pout}".format( + string=string, pout=pandoc_output + ) + ) + + +def run_tex_tests(args, fmt): + """ + Run same tests for latex writers. + + ``args`` (list of str) + The base list of arguments to forward to pandoc. + + ``fmt`` (str) + The format is assumed to be either 'latex' or 'beamer'. + """ + def verify(test_name, md, string): + """Run pandoc, ensure fragile, and string in output.""" + output = run_pandoc(args + ["-t", fmt], md) + if fmt == "beamer": + ensure_fragile(test_name, output) + else: # latex writer + ensure_not_present(test_name, "fragile", output) + ensure_present(test_name, string, output) + # if `nil` is present, that likely means a problem parsing the metadata + ensure_not_present(test_name, "nil", output) + + ############################################################################ + # CodeBlock tests. # + ############################################################################ + begin_minted = r"\begin{{minted}}[{attrs}]{{{lang}}}" + verify( + "[code-block] default", + code_block, + begin_minted.format(attrs="autogobble", lang="cpp") + ) + verify( + "[code-block] no_default_autogobble", + textwrap.dedent(''' + --- + minted: + no_default_autogobble: true + --- + {code_block} + ''').format(code_block=code_block), + begin_minted.format(attrs="", lang="cpp") + ) + verify( + "[code-block] default block language is 'text'", + code_block.replace("{.cpp}", ""), + begin_minted.format(attrs="autogobble", lang="text") + ) + verify( + "[code-block] user provided default_block_language", + textwrap.dedent(''' + --- + minted: + default_block_language: "haskell" + --- + {code_block} + ''').format(code_block=code_block.replace("{.cpp}", "")), + begin_minted.format(attrs="autogobble", lang="haskell") + ) + verify( + "[code-block] user provided block_attributes", + textwrap.dedent(''' + --- + minted: + block_attributes: + - "showspaces" + - "space=." + --- + {code_block} + ''').format(code_block=code_block), + begin_minted.format( + attrs=",".join(["showspaces", "space=.", "autogobble"]), + lang="cpp" + ) + ) + verify( + "[code-block] user provided block_attributes and no_default_autogobble", + textwrap.dedent(''' + --- + minted: + no_default_autogobble: true + block_attributes: + - "style=monokai" + - "bgcolor=monokai_bg" + --- + {code_block} + ''').format(code_block=code_block), + begin_minted.format( + attrs=",".join(["style=monokai", "bgcolor=monokai_bg"]), lang="cpp" + ) + ) + verify( + "[code-block] attributes on code block", + code_block.replace( + "{.cpp}", "{.cpp .showspaces bgcolor=tango_bg style=tango}" + ), + begin_minted.format( + attrs=",".join([ + "showspaces", "bgcolor=tango_bg", "style=tango", "autogobble" + ]), + lang="cpp" + ) + ) + verify( + "[code-block] attributes on code block + user block_attributes", + textwrap.dedent(''' + --- + minted: + block_attributes: + - "showspaces" + - "space=." + --- + {code_block} + ''').format( + code_block=code_block.replace( + "{.cpp}", "{.cpp bgcolor=tango_bg style=tango}" + ) + ), + begin_minted.format( + attrs=",".join([ + "bgcolor=tango_bg", + "style=tango", + "showspaces", + "space=.", + "autogobble" + ]), + lang="cpp" + ) + ) + verify( + "[code-block] traditional fenced code block", + code_block.replace("{.cpp}", "cpp"), + begin_minted.format(attrs="autogobble", lang="cpp") + ) + verify( + "[code-block] non-minted attributes not forwarded", + code_block.replace("{.cpp}", "{.cpp .showspaces .hello}"), + begin_minted.format( + attrs=",".join(["showspaces", "autogobble"]), lang="cpp" + ) + ) + + ############################################################################ + # Inline Code tests. # + ############################################################################ + mintinline = r"\mintinline[{attrs}]{{{lang}}}" + verify( + "[inline-code] default", + inline_code, + mintinline.format(attrs="", lang="cpp") + ) + verify( + "[inline-code] default language is text", + inline_code.replace("{.cpp}", ""), + mintinline.format(attrs="", lang="text") + ) + verify( + "[inline-code] no_mintinline globally turned off", + textwrap.dedent(''' + --- + minted: + no_mintinline: true + --- + {inline_code} + ''').format(inline_code=inline_code), + r"\texttt{#include }" + ) + verify( + "[inline-code] .no_minted class bypasses single inline code element", + inline_code.replace("{.cpp}", "{.cpp .no_minted}"), + r"\texttt{#include }" + ) + verify( + "[inline-code] default inline language is 'text'", + inline_code.replace("{.cpp}", ""), + mintinline.format(attrs="", lang="text") + ) + verify( + "[inline-code] user provided default_inline_language", + textwrap.dedent(''' + --- + minted: + default_inline_language: "haskell" + --- + {inline_code} + ''').format(inline_code=inline_code.replace("{.cpp}", "")), + mintinline.format(attrs="", lang="haskell") + ) + verify( + "[inline-code] user provided inline_attributes", + textwrap.dedent(''' + --- + minted: + inline_attributes: + - "showspaces" + - "space=." + --- + {inline_code} + ''').format(inline_code=inline_code), + mintinline.format( + attrs=",".join(["showspaces", "space=."]), lang="cpp" + ) + ) + verify( + "[inline-code] attributes on inline code", + inline_code.replace( + "{.cpp}", "{.cpp .showspaces bgcolor=tango_bg style=tango}" + ), + mintinline.format( + attrs=",".join(["showspaces", "bgcolor=tango_bg", "style=tango"]), + lang="cpp" + ) + ) + verify( + "[inline-code] attributes on inline code + user inline_attributes", + textwrap.dedent(''' + --- + minted: + inline_attributes: + - "showspaces" + - "space=." + --- + {inline_code} + ''').format( + inline_code=inline_code.replace( + "{.cpp}", "{.cpp bgcolor=tango_bg style=tango}" + ) + ), + mintinline.format( + attrs=",".join([ + "bgcolor=tango_bg", + "style=tango", + "showspaces", + "space=." + ]), + lang="cpp" + ) + ) + verify( + "[inline-code] non-minted attributes not forwarded", + inline_code.replace("{.cpp}", "{.cpp .showspaces .hello}"), + mintinline.format(attrs="showspaces", lang="cpp") + ) + + +def run_html_tests(args): + """ + Run tests with an html5 writer to make sure minted commands are not used. + Also make sure minted specific attributes are indeed stripped. + + ``args`` (list of str) + The base list of arguments to forward to pandoc. + """ + def verify(test_name, md, attrs=[]): + """Verify minted and any strings in attrs not produced""" + output = run_pandoc(args + ["-t", "html5"], md) + ensure_not_present(test_name, "mint", output) + ensure_not_present(test_name, "fragile", output) + if attrs: + for a in attrs: + ensure_not_present(test_name, a, output) + # if `nil` is present, that likely means a problem parsing the metadata + ensure_not_present(test_name, "nil", output) + + verify(r"[html] no \begin{minted}", code_block) + verify(r"[html] no \mintinline", inline_code) + verify( + r"[html] no \begin{minted} or \mintinline", + "{code_block}\n\n{inline_code}".format( + code_block=code_block, inline_code=inline_code + ) + ) + verify( + "[html] code block minted specific attributes stripped", + code_block.replace( + "{.cpp}", + "{.cpp .showspaces space=. bgcolor=minted_bg style=minted}" + ), + ["showspaces", "space", "bgcolor", "style"] + ) + verify( + "[html] inline code minted specific attributes stripped", + inline_code.replace( + "{.cpp}", + "{.cpp .showspaces space=. bgcolor=minted_bg style=minted}" + ), + ["showspaces", "space", "bgcolor", "style"] + ) + + +if __name__ == "__main__": + # Initial path setup for input tests and lua filter + this_file_dir = os.path.abspath(os.path.dirname(__file__)) + minted_lua = os.path.join(this_file_dir, "minted.lua") + if not os.path.isfile(minted_lua): + sys.stderr.write("Cannot find '{minted_lua}'...".format( + minted_lua=minted_lua + )) + sys.exit(1) + + args = ["--fail-if-warnings", "--lua-filter", minted_lua] + run_tex_tests(args, "beamer") + run_tex_tests(args, "latex") + run_html_tests(args) diff --git a/minted/sample.md b/minted/sample.md new file mode 100644 index 00000000..3758f19a --- /dev/null +++ b/minted/sample.md @@ -0,0 +1,108 @@ +--- +title: Pandoc Minted Sample +# NOTE: If you want to use `\definecolor` commands in your `header-includes` +# section, setting `colorlinks: true` will `\usepackage{xcolor}` which is needed +# for `\definecolor`. You can alternatively `\usepackage{xcolor}` explicitly in +# in the `header-includes` section if you do not want everything else that +# `colorlinks: true` will bring in. See `pandoc -D latex` output to see +# everything that `colorlinks: true` will do _in addition_ to including xcolor. +colorlinks: true +header-includes: + # Include the minted package, set global style, define colors, etc. + - "\\usepackage{minted}" + - "\\usemintedstyle{tango}" + - "\\definecolor{tango_bg}{rgb}{0.9725,0.9725,0.9725}" + - "\\definecolor{monokai_bg}{rgb}{0.1529,0.1569,0.1333}" + # NOTE: comment out these last three and recompile to see the italics used + # by default for the `tango` style. + # Prevent italics in the `minted` environment. + - "\\AtBeginEnvironment{minted}{\\let\\itshape\\relax}" + # Prevent italics in the `\mintinline` command. + - "\\usepackage{xpatch}" + - "`\\xpatchcmd{\\mintinline}{\\begingroup}{\\begingroup\\let\\itshape\\relax}{}{}`{=latex}" +minted: + block_attributes: + - "bgcolor=tango_bg" +--- + +## Inline Code in Pandoc + +- Raw inline code: + + ```md + `#include ` + ``` + + \vspace*{-3ex} produces: `#include ` + +- Apply just a lexer: + + ```md + `#include `{.cpp} + ``` + + \vspace*{-3ex} produces: `#include `{.cpp} + +- Change the background color and highlighting style: + + ```{.md fontsize=\scriptsize} + + `#include `{.cpp bgcolor=monokai_bg style=monokai} + ``` + + \vspace*{-3ex} produces: + `#include `{.cpp bgcolor=monokai_bg style=monokai} + + - Must **always** include language (`.cpp` here) **first**, always! + +## Inline Code Bypasses + +- Want the regular teletype text? Specify **both** the lexer class name and one + additional class `.no_minted`. + + ```{.md} + + `no minted`{.text .no_minted} + ``` + + \vspace*{-3ex} produces: `no mintinline`{.text .no_minted} vs `with mintinline` + + - Inspect generated code, the PDF output is indistinguishable. + +- Alternatively, you can set `no_mintinline: true`{.yaml style=paraiso-light} to prevent the filter + from emitting _any_ `\mintinline`{.latex} calls. + - If you don't need syntax highlighting on your inline code elements, this may + greatly improve compile times for large documents. + + +## Code Blocks + +- Use the defaults, but still supply the lexer: + + ```bash + echo "Hi there" # How are you? + ``` + + \vspace*{-3ex} produces + + ```bash + echo "Hi there" # How are you? + ``` + + \vspace*{-3ex} + +- As with inline code, you can change whatever you want: + + ```{.bash bgcolor=monokai_bg style=monokai} + echo "Hi there" # How are you? + ``` + + \vspace*{-3ex} produces + + ```{.bash bgcolor=monokai_bg style=monokai} + echo "Hi there" # How are you? + ``` + + \vspace*{-3ex} + + - Must **always** include language (`.bash` here) **first**, always!