Variable preprocessor

[rivertam]

During a discussion for the Amethyst book, we realized we wanted a way to put in doc badges dynamically dependent on the current building version of the Amethyst library.

Is there a way to read from some sort of global key value store (such as environment variables or the configuration) while rendering the markdown? I believe a preprocessor could accomplish this task. Something like

[![develop docs](https://img.shields.io/badge/docs-develop-blue.svg)](#[docsRoot]/amethyst/ecs/prelude/trait.Component.html) [![master docs](https://img.shields.io/badge/docs-master-blue.svg)](#[docsRoot]/amethyst/ecs/prelude/trait.Component.html)

(the idea being #[variableName] would be the way to access a variable -- probably a terrible particular example)

If there is such a way to do this, I don't think it's documented.

[mattico]

So in-markdown templates, essentially? No I do not think there is currently a way to do this, but a preprocessor would work well.

[rivertam]

@mattico Yeah, exactly. Do you think it would make sense to be a core preprocessor for mdBook, or would it necessarily be a custom solution? I foresee this being a common need for things exactly like what I'm describing, but I'm much less tuned in to the needs of the average user here.

[mattico]

I've never seen a need for it, but since the core audience of mdBook is programmers I think most would understand and appreciate the feature.

I do think we'll want a more modular interface for configuring preprocessors at some point, so that we can add them more easily for niche use cases.

[rbuckland]

With @Michael-F-Bryan 's PR for #658, it supports

[preprocesor.variable-templater]
my_custom_config = "foo bar"

My suggestion would be to pick something off the shelf that can do this - and what instantly comes to mind is ``{{ templating }}` - The Python Jinja format .. which is also common with "go" templates.

Rust has it's own - https://github.com/djc/askama

So perhaps a preprocessor that, at a minimum, provides the "config" of the preprocessor, plus ENV_VARS into the conversion.

Seems simple enough.

it would just need to co-exist with the {{ #import filepath args }} syntax. but that "should" be okay if we pre-process links, always before {{ template_vars }}

[jeffvandyke]

As a related feature to having variables, a way to put conditionals in markdown would allow different outputs from the same source based on environment variables or configuration. A use case that's relevant to our company would be to bundle a help manual in a web application on multiple different embedded devices with a few different configuration options. It would be ideal to be able to conditionally render words or sections of markdown, or conditionally include different pages in the SUMMARY.md file.

I think a Handlebars preprocessor would be a valid and coherent way to make this work (seeing how mdBook uses Handlebars already)? Handlebars has a form capable of conditionally rendering words or paragraphs of markdown, e.g.: {{#if condition}} Markdown here {{else}} other markdown {{/if}}.

[jeffvandyke]

Askama vs Handlebars... thoughts? Askama will be much faster (benchmarks), Handlebars might be more well known, but Askama isn't difficult... I think having a templating language for variables and conditionals, is important for any static site generator.

[tglman]

hi All,
I just implemented a variable preprocessor, based on regular expression and without anything complex, here it is: https://gitlab.com/tglman/mdbook-variables, I'm commenting here to get some early feedback before publishing it, so far it just replace {{variable_name}} with a value configured in the book.toml.

Looking for some feedback.
Regards

[avitex]

Adding to this, if you like Tera templates, see https://github.com/avitex/mdbook-tera

[Michael-F-Bryan]

so far it just replace {{variable_name}} with a value configured in the book.toml.

Have you thought about environment variables? I feel like people could want to use that to inject git hashes when publishing a document and wanting to link back to the source on GitHub.

[tglman]

Hi @Michael-F-Bryan,

Yes that is not a bad idea i will have a quick look how that could be done.

Regards

[tglman]

Hi @Michael-F-Bryan,

Have a look: https://gitlab.com/tglman/mdbook-variables/-/commit/2d628ce6ef5c243659a152ef2770ef42eeb4608f not sure though about precedence and how to enabled that, feel free to open an issue on that repository with suggestions.

Bye

[Hoffenbar]

Neither mdbook-variables, nor mdbook-tera have binaries available....... do I really have to install rust..?

[ihojmanb]

Hi @tglman! is there a way to use the variables preprocessor inside de SUMARRY.md? I'm trying to use variables for file paths to respect DRY pattern but mdbook doesn't seem to like it.

[Michael-F-Bryan]

Hi @tglman! is there a way to use the variables preprocessor inside de SUMARRY.md? I'm trying to use variables for file paths to respect DRY pattern but mdbook doesn't seem to like it.

This probably isn't possible. Preprocessors are only run on the chapters of a book once they've been loaded into memory, by which time the SUMMARY.md has already been parsed (it's how we find the chapters to give the preprocessor, after all).

[tglman]

Hi,

Yep as @Michael-F-Bryan said is not possible, and to make it possible is actually needed a bit of architecture change in mdBook itself.

Bye

[ihojmanb]

thanks both of you for clarifying that. Makes me sad, though.

[avitex]

Neither mdbook-variables, nor mdbook-tera have binaries available....... do I really have to install rust..?

Just noting this is now fixed in mdbook-tera from v0.4.2 onwards. Binary releases are now published.

[Wilfred]

A preprocessor is transforming the JSON of the book, and jq .[1] is a valid minimal preprocessor. I was able to include the cargo version in my mdbook by writing the following preprocessor:

#!/bin/bash

DFT_VERSION=$(cargo read-manifest | jq -r .version)

jq .[1] | jq '.sections[0].Chapter.content |= sub("DFT_VERSION_HERE"; "'$DFT_VERSION'")'

See Wilfred/difftastic@80429e9 for the full configuration I used.

[sgoudham]

Hi all, just adding my own two cents to this thread.

I also wanted to have templates included in my code as I found that I wanted a solid footer included on all pages. From that, I designed mdbook-template which is a file-based config and very similar to how the current {{#include}} works.

It's by no means perfect but allows for default values and dynamic variables to be passed too 👍