TechnicalMarkdown

A markdown setup for technical documents, reports, theses & papers.

Quick Intro

This is a markdown setup demonstrating the power and use of markdown for technical documents:

• fully automated conversion sequence using gradle + pandoc such that exporting (Content.md) is done in the background:

  • export to PDF with pandoc to xelatex using latexmk See Output
  • export to HTML with pandoc to html See Output
  • [todo] export to PDF with pandoc to html then to chrome with pupeteer

• pandoc filters for different AST (abstract syntax tree) conversions:

  • own filters with panflute [doc]
  • --crosscite [doc] for citing
  • pandoc-crossref [doc] for cross referencing
  • pandoc-include-files [doc] for file transclusion

• Full-fledged VS Code setup to write and style your document in one of the best IDEs.

Quick-Start

Execute the following in a shell:

./gradlew -t build-html

This will build the HTML output from its markdown main file.

Rational

Pandoc is awesome and the founder John MacFarlane develops pandoc in a meticulous and principled style. The documentation is pretty flawless and the community (including him) is really helpful. That is why we rely heavily on pandoc.

1. We target the output formats html5 and latex, because

   • HTML can be viewed in all browsers and web standards such as CSS3 etc. have become a major advantage and enables ridiculuous dynamic, interactive styling. Collapsable table of contents is just the beginning.
   • LaTeX enables to produce high quality output PDF (xelatex). Every proper book and distributed PDF is written and set in LaTeX.

2. The orchestration around calling pandoc is basically only a file watcher gradle which calls pandoc on file changes. We want as little as possible different tools to achieve the above output formats. That also means we do not want to have lots of pre- and post-processing tasks aside from running pandoc. The main goal is, that users can write markdown as a first-party solution with some enhanced features enabled by pandoc itself. Writting technical documents should become a breeze.

3. The common agreement in the industry about using M$ Office for writting technical documentations as demonstrated here, is considered the most complete and utter bullshit you can adhere to. Certainly employees mostly must obey. The common argument is "people need to exchange documents and work on it". Because people need to focus on the utter shitty formatting and WYSIWYG workaround experiences, a lot of time and money is spent which gets never debated.

   It's about high time to turn into a direction which will likely become the standard. Technical writters should really focus on the content they write and not focus on styling quirks and tricks.

4. Every technical document writter probably knows about source code management (git). There you go with proper team work.

Dependencies

Gradle

For the Gradle build tool you need a working Java runtime. On Linux and macOS you can do:

brew install java

You should not need to install Gradle, since everything is setup by the checked-in gradlew Gradle wrapper.

Pandoc

Install pandoc (>= 2.9.2.1, testet with 2.11.0.4)

For Linux and macOs:

export HOMEBREW_NO_INSTALL_CLEANUP=1
cd Homebrew/Library/Taps/homebrew/homebrew-core &&
    git checkout 36b6c2d8cd71580a4fd3055375f87a8c52cd5846~20 && # installs 2.9.2.1
    HOMEBREW_NO_AUTO_UPDATE=1 brew install pandoc pandoc-citeproc pandoc-crossref &&
    brew install pandoc pandoc-citeproc pandoc-crossref # installs 2.10.1

For Windows

choco install pandoc
git clone https://github.com/gabyx/chocolatey-packages.git@patch-1 temp
cd temp && choco install ./pandoc-crossref/pandoc-crossref.nuspec

Python

Install a recent python3 (>= 3.6) and the following packages

pip3 install -r .requirements

The best way is to setup a python environment python venv since pandoc. The VS Code config python.pythonPath path needs to be set.

The VS Code tasks get the ${config:python.pythonPath} directly as an argument and modify the environement and pandoc will use the right python when launching the filters.

You can also start VS Code like this to have the proper python enabled:

# Activate your python env.
source ~/.env/myPython3.8Env/activate
# Start code
cd TechnicalMarkdown && code -n .

You can also use the ignored .envrc file with direnv.

Building and Viewing

Run the following tasks defined in tasks.json from VS Code or use the following shell commands:

• Show HTML Output: Serves the HTML for preview in a browser with autoreload:

  ./gradlew -t view-html

• Convert Markdown -> HTML: Runs the markdown conversion with Pandoc (html) continuously:

  ./gradlew -t build-html

  • The conversion with pandoc applies the following filters in defaults.
  • The HTML output can be inspected in Content.html.

• Convert Markdown -> PDF: Runs the markdown conversion with Pandoc (latexmk and xelatex) continuously:

  ./gradlew -t build-pdf-tex

  • The conversion with pandoc applies the following filters in defaults.
  • The PDF output can be inspected in Content.pdf.
  • The LaTeX output can be inspected in build/output-tex/input.tex.

• Convert Markdown -> Jira: Runs the markdown conversion to Jira (experimental) with Pandoc continuously:

  ./gradlew -t build-jira

  • The conversion with pandoc applies the following filters in defaults.
  • The Jira output can be inspected in Content.jira.

Editing Styles

HTML

You can edit the main.less file to change the look of the markdown. Edit the main.less file to see changes in the conversion from Content.md.

LaTeX

The following templates are responsible for the LaTeX output:

• Template.tex : The main template.
• Header.tex : The class, packages and styles defining the document, included by the main template with include-in-header in pandoc-latex.yaml

Debugging

There is a debug configuration in launch.json for both the HTML and the PDF export.

Pandoc Filters

Pandoc filters are harder to debug. There is an included unix-like tee.py filter which can be put anywhere into the filter chain as needed, to see the AST JSON output in the folder build/pandoc-filter-out (see dev.py for adjustments). The filter teeStart.py first clears all output before doing the same as tee.py. Uncomment the tee.py filters in pandoc-filters.yaml.

Issues

Panflute [done]

Using pandoc >=2.10 we have more types and AST changes in

• jgm/pandoc#1024,
• jgm/pandoc#6277,
• 2.10

meaning that also the python library panflute needs to be supporting this:

• Issue :

Transclude: Relative file paths

So far relative paths are not yet supported in pandoc-indluce-files.lua filter.

• Issue :

Table Issues

• Wrong format for latex: Issue : -> Update to next version.

Todo

• Add CI.
• Add tests.
• Add prince conversion to PDF.