Allow for substitutions inside link targets

[choldgraf]

In jupyterhub/zero-to-jupyterhub-k8s#1968 (comment), @consideRatio mentioned that it'd be useful to be able to use substitutions inside things like inline elements. In this case, for the target of a link. I believe he wanted to do something like:

myst:
  substitutions:
    version: v3
---

Here is an [inline link with the target version](https://myorg.com/{{ version }}/docs)

I don't think that this is currently possible, but I could see it being useful (e.g., if you're linking to the same external documentation that has a regularly-changing root, such as one based on a version).

[chrisjsewell]

Ah no this is not how it works, its a bit more nuanced than that:

Firstly the substitutions have to be identified by the markdown-it plugin and converted to a specific syntax token, they can't just be placed anywhere (as it says in the documentation)

Secondly they don't just "copy/paste" the text from the substitution definition to the reference; the text from the definition is parsed to AST in isolation (after jinja filters have been applied), then injected into the documents AST.

This, for example, allows for you to inject directives into inline text, e.g. to create this crazy table from https://myst-parser.readthedocs.io/en/latest/using/syntax-optional.html#substitutions-with-jinja2:

if you just copied:

```{image} path
:width: 100px
```

into:

| -------- | - |
| {{key1}} | a |

you would end up with:

| -------- | - |
| ```{image} path
:width: 100px
``` | a |

which obviously would not be recognised or rendered as a table.

If you wanted this, its more of just a plain jinja pre-conversion of the source file, that has no concept of the actual document.

or you could do:

---
substitutions:
  version: v3
  doc_link: "[inline link with the target version](https://myorg.com/VERSION/docs)"
---

Here is an {{ doc_link | replace('VERSION', version) }}

(I tested this and it works)

or in the conf.py

version = "v3"
myst_substitutions = {
  "doc_link": f"[inline link with the target version](https://myorg.com/{version}/docs)"
}

[choldgraf]

ah that is pretty clever! cc @consideRatio - you should check out the pattern described here: #279 (comment)

I suspected that this wouldn't be solvable on the code side but good to know there's a workaround.

Do you think it is worth documenting this pattern? I can open a PR if so.

[chrisjsewell]

Do you think it is worth documenting this pattern? I can open a PR if so.

yeh if there is anything you think would help people, go for it.
I guess the important concept to convey is that the substitutions are not just injecting the text, they are injecting the AST created from the text (after jinja resolution).

[consideRatio]

Thank you both @choldgraf and @chrisjsewell!

I ended up with the full link substitution, where markdown for the entire link was decided in conf.py (here). In other words, I ended up doing what @chrisjsewell suggested with the example...

or in the conf.py

version = "v3"
myst_substitutions = {
  "doc_link": f"[inline link with the target version](https://myorg.com/{version}/docs)"
}

EDIT: Thank you a lot @chrisjsewell for describing this, it was a missing piece of understanding for me:

Firstly the substitutions have to be identified by the markdown-it plugin and converted to a specific syntax token, they can't just be placed anywhere (as it says in the documentation)

Secondly they don't just "copy/paste" the text from the substitution definition to the reference; the text from the definition is parsed to AST in isolation (after jinja filters have been applied), then injected into the documents AST.

[chrisjsewell]

I've added a specific section on URL substitutions: https://myst-parser.readthedocs.io/en/latest/using/syntax-optional.html#substitutions-and-urls

Note, I did consider if substitutions could be used directly in URLs, but decided it would be too complex/error-prone with the Markdown parsing, e.g. spaces in URLs are not allowed so <https://{{ key }}> would fail, and also with HTML escaping means https://{{key}} is converted to https://%7B%7Bkey%7D%7D.com and so would then have to be unescaped before running jinja and re-escaped after.

[consideRatio]

Thanks for your work and follow up on this @chrisjsewell!! 🎉 ❤️

[mforbes]

@chrisjsewell Is there any way to have Jinja use f-strings, or something similar? Your example in the docs:

{{ '[a link](https://{}.com)'.format(key4) }}

can be written

{{ '[a link](https://{key4}.com)'.format(**env.config.myst_substitutions) }}

While this is rather hideous, I am wondering if there is a way of making this simpler: Could jinja support f-strings?

{{ f'[a link](https://{key4}.com)' }}

or at maybe we could add a custom filter subs:

{{ '[a link](https://{key4}.com)'|subs }}

This works with a one-line modification to docutils_renderer.DocutilsRenderer.render_substitution

        env.filters["subs"] = lambda value: value.format(**variable_context)

Is this something I can do with conf.py?