Allow Jekyll Frontmatter

[krono]

Jekyll Frontmatter (A YAML block before actual markdown text) is commonly used with github pages and markdown.

It would be great to be able to

• have Frontmatter in included Markdown files and maybe
• set a callback/command to process the contents.

---

Background: I have some people giving me tex files that includes things like authors, title, etc. I really would like to have that as markdown, but cannot deal with the metadata.

E.g., currently I get

\author{John Doe}
\title{The Theory of Everything}
\maketitle
\begin{abstract}
foo
\end{abstract}

Lorem \emph{ipsum} ...

But I would like

---
author: John Doe
title: The Theory of Everything
abstract: >
  foo
---

Lorem *ipsum* ...

And then turn the latter into the forme

[Witiko]

Hello,

you can abuse the existing syntax to achieve a similar effect:

\documentclass{article}
\usepackage{markdown}
\def\markdownRendererHeadingOne#1{\title{#1}}
\def\markdownRendererHeadingTwo#1{\author{#1}}
\def\markdownRendererBlockQuoteBegin{%
  \maketitle
  \begin{abstract}}
\def\markdownRendererBlockQuoteEnd{%
  \end{abstract}%
  % The end of the front matter.
  \def\markdownRendererHeadingOne##1{\markdownRendererHeadingOnePrototype{##1}}%
  \def\markdownRendererHeadingTwo##1{\markdownRendererHeadingTwoPrototype{##1}}%
  \def\markdownRendererBlockQuoteBegin{\markdownRendererBlockQuoteBeginPrototype}%
  \def\markdownRendererBlockQuoteEnd{\markdownRendererBlockQuoteEndPrototype}}
\begin{document}
\begin{markdown}
# The Theory of Everything
## John Doe
> foo

# Level 1 heading
## Level 2 heading
Lorem *ipsum* ...
\end{markdown}
\end{document}

[Witiko]

Of course, you will need proper YAML blocks if you use your Markdown documents for multi-target publishing. Sorry to say that the package currently does not support them. If you have a partial working implementation, I would be happy to help you finalize it and merge it.

You may also want to consider using external tools that support YAML blocks and produce LaTeX markup, such as Pandoc or MultiMarkdown. The downside is that you do not get such a nice interface to control the form of the resulting documents with these tools.

[TomBener]

Thanks for creating the amazing package!

I wonder whether the package could convert the yaml block to the proper PDF metadata generated by hyperref at the moment? for example, yaml block in markdown file is like:

title: bah1
author: bah2
subject: bah3
...

want to convert to LaTeX👇:

pdftitle = {bah1},
pdfauthor = {bah2},
pdfsubject = {bah3},
...

Alternatively, I just want to ignore the yaml when compiling the LaTeX file. Now, the markdown package regards the yaml block as the normal body. As the yaml block is essential for some purposes, I don't want to delete it but just want the package to neglect it? Is it possible to make it? Thanks!

[TeXhackse]

I am not quite sure if that might help, but @krono told me to give it a try.

I usually used expl3 to parse jekyll headers and saw that issue last week. I made a rough draft, but without having spent any time on getting the structure of the package and abused the fenceCode parser (which is ofc not ideal but similar to the structure) to parse the contents of jekyll frontmatter to a keyval list. That one could easily be used to do whatever the user is expecting.

Since it's just quick and dirty I did not yet create a PR but in case you find it helpful I still can add one. Otherwhise you can find it in my fork: https://github.com/TeXhackse/markdown/tree/jekyll-header

[Witiko]

@TeXhackse Interesting, thank you for your work! I glanced through your draft and it seems that you are converting the jekyll frontmatter to a comma-separated list of key=value pairs that can be fed e.g. to the keyval package? Can you give a short example of how you envision this would be used by the user (i.e. a minimal definition of \markdownRendererSetJekyllData)?

P.S.: You should not need to enable options.fencedCode. The parsers.fencehead, fencebody, and fencetail will be defined even when options.fencedCode is false.

[TeXhackse]

thanks for the P.S. I did not use enough time to have a closer look it was rather an idea and I wanted to check it :) But I can try to find time to clean it up anyway.

Yes, though I'd not use keyval but expl3 but both should be possible. Depending on the usecase I'd save the data to a property list or to macros like \@title sometimes with an optional prefix.

For testing I used expl3 which was the fastest:

\ExplSyntaxOn
\tl_new:N \l_ptx_jekyll_data_prefix_tl
\tl_set:Nn \l_ptx_jekyll_data_prefix_tl {}

\keys_define:nn {ptx/markdown/jekyll} {
	unknown .code:n = \cs_set:cpn {\l_ptx_jekyll_data_prefix_tl@\l_keys_key_str} {#1}
}

\renewcommand*{\markdownRendererSetJekyllData}[1]{
	\keys_set:nn {ptx/markdown/jekyll} {#1}
}
\ExplSyntaxOff

But one could also define it to use predefined keyval keys:

\renewcommand*{\markdownRendererSetJekyllData}[1]{%
	\setkeys{<module>}{#1}%
}

P.S.: I know that there's a keyval syntax for that redefinition, this was just copy paste from another project.

[Witiko]

Perhaps the default could be just keyval with predefined title -> \title{#1}, author -> \author{#1} and users could easily add more fields by adding more keyval keys?

[krono]

Thats really nice.
I could define \markdownRendererSetJekyllData to use the KOMA-Family-option mechanism that way…

[Witiko]

However, I am a little sceptical about the ease of parseability outside LaTeX. The Markdown package also supports plain TeX and the use should be straightforward. Therefore it might be preferrable to produce a single \markdownRendererSetJekyllData command for each data item, so that the following document:

---
author: John Doe
title: The Theory of Everything
abstract: >
  foo
---

would become this TeX output:

\markdownRendererSetJekyllData{author}{John Doe}
\markdownRendererSetJekyllData{title}{The Theory of Everything}
\markdownRendererSetJekyllData{abstract}{foo}

This way, we can still set up default handling with keyval in LaTeX for the ease of adding more key definitions, but we are not making life difficult for the plain TeX folks.

[TeXhackse]

Oh the TeX way can easily be added, give me a second.

[krono]

Also I would propose
\markdownRendererSetFrontmatterData
instead of
\markdownRendererSetJekyllData

[Witiko]

Or perhaps the more declarative \markdownRendererFrontmatterItem. We could even add the surrounding Begin and End commands, so that the front matter is conceptually the same as definition lists, numbered lists and bullet lists:

\markdownRendererFrontmatterBegin
\markdownRendererFrontmatterItem{author}{John Doe}
\markdownRendererFrontmatterItem{title}{The Theory of Everything}
\markdownRendererFrontmatterItem{abstract}{foo}
\markdownRendererFrontmatterEnd

This way, you can even add thing such as \markdownRendererFrontmatterBegin -> \frontmatter and \markdownRendererFrontmatterEnd -> \maketitle\mainmatter for the book class for proper display.

[TeXhackse]

So for TeX maybe the comma list is not a good idea, but one could convert every item to a single macro which would not change the use of keyval packages.

\def\helper #1 = #2 \relax {
	\expandafter\def\csname @#1\endcsname {#2}
}

one would have to wrap it to cleanup the expansion but that should not be an issue.

[TeXhackse]

Actually I don't care how can reference the data. I usually only need the syntax to generic define things. And due to expl3 KeyVal is the simplest to use. But you are right, maybe some other Syntax might be easier for others.

[Witiko]

So for TeX maybe the comma list is not a good idea, but one could convert every item to a single macro which would not change the use of keyval packages.

\def\helper #1 = #2 \relax {
	\expandafter\def\csname @#1\endcsname {#2}
}

one would have to wrap it to cleanup the expansion but that should not be an issue.

@TeXhackse That is definitely true, but we don't lose anything by splitting the key and value to separate parameters and it makes life a little easier for everybody. Just brainstorming here.

[Witiko]

I have added this feature to the 2.10.0 milestone. If you'd like to clean this up into a PR, that would be appreciated; otherwise, I can take over from here. I will also be writing a TUGboat article about the new features; we could share authorship if you'd like.

[TeXhackse]

I will try to do so, but won't manage to have another look at it before the weekend.

[Witiko]

That's no issue at all. We will also want to document the additions, but I can do that.

[TeXhackse]

I cleaned if up a bit and tried to follow the documentation structure. I implemented the two-argument variant, because as we realized this might be easier for average users. One could easily use it to fill a keyval structure anyway.

It's still not clean so I just opened a draft PR and will continue as much as possible, but due to a lack of time that might be not that helpful at all.

[Witiko]

@TeXhackse Thank you. I will take a look and help finish the PR.

[Witiko]

Since we are close to the release of Markdown 2.10.0 and #77 is far from completed, I am thinking of postponing this issue to the next minor version. What do you think?

[TeXhackse]

That's okay for me, I am just sorry I didn't manage to complete it as fast as I was expecting.

I am currently trying to find some free time to continue. I just expected it to be easier (probably we all do, when we don't know the structure of a package ;-) ) So thanks a lot for the hints so far.

[Witiko]

probably we all do, when we don't know the structure of a package ;-)

That's a nice way of putting it. 😄
The package would benefit from splitting the markdown.dtx file into a set of smaller files, each with just a few responsibilities.
If it's difficult for me to contribute new functionality, I can only imagine what new contributors experience.

[kalekje]

I just want to add my "hacky" solution to this: One could pre-read the .md file, store it as a string, stop at the end of the yaml block (by detecting the second ---), and use that string with a lua package like tinyyaml to parse it into a lua table, then use some lua functions to write latex code with tex.print for \def or \pdfinfo, etc.

Thanks for this great package.

[krono]

Woho!