File-level metadata in Markdown? #95

scripting · Sep 17, 2018

I wonder if anyone's thought about a way to add file-level metadata to a Markdown document. In Frontier's website framework, we used a # to delimit a value, something like:

#title "My Test Page"

We have the same thing in HTML and OPML, in the <head> section:

<title>My Test Page</title>

The idea goes all the way back to C on Unix in the 70s:

#include "mymacros.h"

The directives are not part of the rendering. You don't see them when you read the document. But the values are available to software processing the document.

It seems since we're in The Age of JSON, something in JSON, delimited by a # might be appropriate?

#metadata = {
title: "Hello World",
tags: ["fun", "wisdom", "greetings"]
}

This just came up in a project I'm working on, but it's not the first time I've encountered it.

tomraymo · Sep 17, 2018

Jekyll, for example, uses YAML for this purpose. Don't know if that works for your needs.
https://jekyllrb.com/docs/front-matter/

scripting · Sep 17, 2018

@tomraymo -- very interesting. It's a different syntax from what we used in Frontier, they didn't have to appear at the beginning of the document (IIRC). Any line that began with a # was processed as a directive as the page is rendered.

I can already see this is going to be a productive thread. :-)

oevl · Sep 17, 2018

In a similar way to # directives in the Website Framework, the static site generator Hugo uses the concept of "Front Matter" and supports TOML, YAML, and JSON

https://gohugo.io/content-management/front-matter/

kantel · Sep 17, 2018

RubyFrontier does it exactly like the website framework. In fact it is a port of the Frontier's website framework to Ruby, written by Matt Neuburg. My blog entries look like this:

#title "Twine: Change Default Save Location – 20180908"
#maintitle "Twine: Change Default Save Location"

## Worknote: Twine -- Standard-Speicherort ändern

Die Desktop-Version von [Twine](cp^Twine 2) nutzt als (Zwischen-) Speicherplatz der einzelnen Stories
unter MacOS X das Verzeichnis `Dokumente/Twine` (`Documents/Twine`) im Home-Verzeichnis des
Nutzers. Das ist vor allem dann von Nachteil, wenn man zum Beispiel mit mehreren oder einfach auch
nur an unterschiedlichen Rechnern an einer Story arbeitet und diese zum Beispiel via [git](cp^git)
synchronisieren möchte. Unglücklicherweise ist dieses Verzeichnis in der Applikation fest verdrahtet
und eine Änderung auf Applikationsebene leider nicht möglich. …

Whereby #maintitle my own directive is (I use it in my template). To make an optical difference between directitves and headlines I decided to start only with second level headlines. But this is up to you.

RubyFrontier works with different flavours of Markdown. Out of the box it supports the original Markdown (written by John Gruber), Multimarkdown and kramdown. I use kramdown. But it is relative easy to install additional Markdown flavours.

bomberstudios · Sep 17, 2018

Another vote for Jekyll's Front Matter. It is well documented, and even if the YAML format is a bit alien, it's really easy to explain: "oh, it's just like Jekyll Front Matter, here are their docs : )"

karlcow · Sep 18, 2018

MultiMarkdown metadata? Or maybe I misunderstood the initial requirement.

scripting · Sep 18, 2018

I'm so glad I asked this question!

It's really gratifying to see that all CMSes evolved to solve this problem.

There's a natural flow to the design of these things.

It becomes even more interesting when the content lives inside an object database.

And here's the curious thing -- GitHub repos are basically object databases. ;-)

Very much like the structures we use in Frontier to form the storage system for the website framework we developed in Early Web Days.

emk · Sep 18, 2018

I've used Jekyll's YAML metadata before. It's pretty reasonable.

Pros:

If a Markdown file contains Jekyll metadata, GitHub will render the metadata as a nice-looking table at the top of the file.
GitHub Pages supports Jekyll, which makes it easy to publish a directory of static files as a website.
Several other static site rendering tools support Jekyll metadata, so it's become a bit of a de facto standard.
If you want to support lists in your metadata, you can write something like:

tags:
- programming
- thoughts

This will become ["programming", "thoughts"].

Cons:

YAML looks easy to parse, but it's actually fairly complex. There are JavaScript libraries which will do all the work.
Because YAML is relatively complex, YAML parsers have historically had a fair number of security bugs. This really only matters if you allow other people to upload YAML to server.

rahuldave · Sep 19, 2018

Pelican in Python also uses YAML frontmatter. Since jekyll supports it and provides a way for templates to get at the frontmatter, it can be used for many things. For example, I use it to support data downloads for by bayesian stats class. See http://am207.info/homeworks/AM207_HW1.html : the data download is supported in this fashion. Here is the markdown: https://github.com/AM207/2018fall/blob/master/homeworks/AM207_HW1.md

Here the markdown is produced by a custom template from a Jupyter notebook. When i need direct markdown support, I use a markdown editor such as Typora which has explicit YAML support...

kwebble · Sep 20, 2018

In a personal tool I've implemented this similar to HTTP headers, by separating metadata from content by 2 newlines:

key1: value1
another-key: the second value

The content of the page.

The tool knows about some keys, for example labels, and adds them in a consistent way when rendering the page.

scripting · Sep 21, 2018

Question about YAML and the format for "frontmatter" in Jekyll in GitHub.

The frontmatter is delimited by a pair of ---'s.

Can I assume the line delimiter is a \n?

In other words can I search for "---\n"?

Or do I have to allow for the possibility of a \r or a combination of \r\n?

I'm only concerned with how GitHub does this.

bomberstudios · Sep 21, 2018

Here's the REGEXP they use: https://github.com/jekyll/jekyll/blob/4271495fcaad4073b2375b76089041acd16ffb49/lib/jekyll/document.rb#L13

scripting · Sep 22, 2018

I've got YAML working with my GitHub as CMS experiment.

Here's an example of a post so you can see what it looks like.

This is exactly equivalent to using JSON, my server converts back and forth between YAML and JSON, so my app only ever sees the JSON.

BTW this post is a copy of a post I wrote in 1999 about Edit This Page functionality which was then working in Manila, our web CMS.

scripting · Sep 22, 2018

JavaScript code that converts between YAML and JSON and vice versa using js-yaml.

scripting/Scripting-News

File-level metadata in Markdown? #95

File-level metadata in Markdown? #95

scripting commented Sep 17, 2018 •

edited

This comment has been minimized.

tomraymo commented Sep 17, 2018

This comment has been minimized.

scripting commented Sep 17, 2018

This comment has been minimized.

oevl commented Sep 17, 2018

This comment has been minimized.

kantel commented Sep 17, 2018

This comment has been minimized.

bomberstudios commented Sep 17, 2018

This comment has been minimized.

karlcow commented Sep 18, 2018

This comment has been minimized.

scripting commented Sep 18, 2018

This comment has been minimized.

emk commented Sep 18, 2018

This comment has been minimized.

rahuldave commented Sep 19, 2018

This comment has been minimized.

kwebble commented Sep 20, 2018

This comment has been minimized.

scripting commented Sep 21, 2018 •

edited

This comment has been minimized.

bomberstudios commented Sep 21, 2018

This comment has been minimized.

scripting commented Sep 22, 2018

This comment has been minimized.

scripting commented Sep 22, 2018

scripting/Scripting-News

Join GitHub today

File-level metadata in Markdown? #95

Comments

scripting commented Sep 17, 2018 • edited

This comment has been minimized.

tomraymo commented Sep 17, 2018

This comment has been minimized.

scripting commented Sep 17, 2018

This comment has been minimized.

oevl commented Sep 17, 2018

This comment has been minimized.

kantel commented Sep 17, 2018

This comment has been minimized.

bomberstudios commented Sep 17, 2018

This comment has been minimized.

karlcow commented Sep 18, 2018

This comment has been minimized.

scripting commented Sep 18, 2018

This comment has been minimized.

emk commented Sep 18, 2018

This comment has been minimized.

rahuldave commented Sep 19, 2018

This comment has been minimized.

kwebble commented Sep 20, 2018

This comment has been minimized.

scripting commented Sep 21, 2018 • edited

This comment has been minimized.

bomberstudios commented Sep 21, 2018

This comment has been minimized.

scripting commented Sep 22, 2018

This comment has been minimized.

scripting commented Sep 22, 2018

scripting commented Sep 17, 2018 •

edited

scripting commented Sep 21, 2018 •

edited