New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

File-level metadata in Markdown? #95

Open
scripting opened this Issue Sep 17, 2018 · 14 comments

Comments

Projects
None yet
9 participants
@scripting
Owner

scripting commented Sep 17, 2018

I wonder if anyone's thought about a way to add file-level metadata to a Markdown document. In Frontier's website framework, we used a # to delimit a value, something like:

#title "My Test Page"

We have the same thing in HTML and OPML, in the <head> section:

<title>My Test Page</title>

The idea goes all the way back to C on Unix in the 70s:

#include "mymacros.h"

The directives are not part of the rendering. You don't see them when you read the document. But the values are available to software processing the document.

It seems since we're in The Age of JSON, something in JSON, delimited by a # might be appropriate?

#metadata = {
title: "Hello World",
tags: ["fun", "wisdom", "greetings"]
}

This just came up in a project I'm working on, but it's not the first time I've encountered it.

@tomraymo

This comment has been minimized.

Show comment
Hide comment
@tomraymo

tomraymo Sep 17, 2018

Jekyll, for example, uses YAML for this purpose. Don't know if that works for your needs.
https://jekyllrb.com/docs/front-matter/

tomraymo commented Sep 17, 2018

Jekyll, for example, uses YAML for this purpose. Don't know if that works for your needs.
https://jekyllrb.com/docs/front-matter/

@scripting

This comment has been minimized.

Show comment
Hide comment
@scripting

scripting Sep 17, 2018

Owner

@tomraymo -- very interesting. It's a different syntax from what we used in Frontier, they didn't have to appear at the beginning of the document (IIRC). Any line that began with a # was processed as a directive as the page is rendered.

I can already see this is going to be a productive thread. :-)

Owner

scripting commented Sep 17, 2018

@tomraymo -- very interesting. It's a different syntax from what we used in Frontier, they didn't have to appear at the beginning of the document (IIRC). Any line that began with a # was processed as a directive as the page is rendered.

I can already see this is going to be a productive thread. :-)

@oevl

This comment has been minimized.

Show comment
Hide comment
@oevl

oevl Sep 17, 2018

In a similar way to # directives in the Website Framework, the static site generator Hugo uses the concept of "Front Matter" and supports TOML, YAML, and JSON

https://gohugo.io/content-management/front-matter/

oevl commented Sep 17, 2018

In a similar way to # directives in the Website Framework, the static site generator Hugo uses the concept of "Front Matter" and supports TOML, YAML, and JSON

https://gohugo.io/content-management/front-matter/

@kantel

This comment has been minimized.

Show comment
Hide comment
@kantel

kantel Sep 17, 2018

RubyFrontier does it exactly like the website framework. In fact it is a port of the Frontier's website framework to Ruby, written by Matt Neuburg. My blog entries look like this:

#title "Twine: Change Default Save Location – 20180908"
#maintitle "Twine: Change Default Save Location"

## Worknote: Twine -- Standard-Speicherort ändern

Die Desktop-Version von [Twine](cp^Twine 2) nutzt als (Zwischen-) Speicherplatz der einzelnen Stories
unter MacOS X das Verzeichnis `Dokumente/Twine` (`Documents/Twine`) im Home-Verzeichnis des
Nutzers. Das ist vor allem dann von Nachteil, wenn man zum Beispiel mit mehreren oder einfach auch
nur an unterschiedlichen Rechnern an einer Story arbeitet und diese zum Beispiel via [git](cp^git)
synchronisieren möchte. Unglücklicherweise ist dieses Verzeichnis in der Applikation fest verdrahtet
und eine Änderung auf Applikationsebene leider nicht möglich. …

Whereby #maintitle my own directive is (I use it in my template). To make an optical difference between directitves and headlines I decided to start only with second level headlines. But this is up to you.

RubyFrontier works with different flavours of Markdown. Out of the box it supports the original Markdown (written by John Gruber), Multimarkdown and kramdown. I use kramdown. But it is relative easy to install additional Markdown flavours.

kantel commented Sep 17, 2018

RubyFrontier does it exactly like the website framework. In fact it is a port of the Frontier's website framework to Ruby, written by Matt Neuburg. My blog entries look like this:

#title "Twine: Change Default Save Location – 20180908"
#maintitle "Twine: Change Default Save Location"

## Worknote: Twine -- Standard-Speicherort ändern

Die Desktop-Version von [Twine](cp^Twine 2) nutzt als (Zwischen-) Speicherplatz der einzelnen Stories
unter MacOS X das Verzeichnis `Dokumente/Twine` (`Documents/Twine`) im Home-Verzeichnis des
Nutzers. Das ist vor allem dann von Nachteil, wenn man zum Beispiel mit mehreren oder einfach auch
nur an unterschiedlichen Rechnern an einer Story arbeitet und diese zum Beispiel via [git](cp^git)
synchronisieren möchte. Unglücklicherweise ist dieses Verzeichnis in der Applikation fest verdrahtet
und eine Änderung auf Applikationsebene leider nicht möglich. …

Whereby #maintitle my own directive is (I use it in my template). To make an optical difference between directitves and headlines I decided to start only with second level headlines. But this is up to you.

RubyFrontier works with different flavours of Markdown. Out of the box it supports the original Markdown (written by John Gruber), Multimarkdown and kramdown. I use kramdown. But it is relative easy to install additional Markdown flavours.

@bomberstudios

This comment has been minimized.

Show comment
Hide comment
@bomberstudios

bomberstudios Sep 17, 2018

Another vote for Jekyll's Front Matter. It is well documented, and even if the YAML format is a bit alien, it's really easy to explain: "oh, it's just like Jekyll Front Matter, here are their docs : )"

bomberstudios commented Sep 17, 2018

Another vote for Jekyll's Front Matter. It is well documented, and even if the YAML format is a bit alien, it's really easy to explain: "oh, it's just like Jekyll Front Matter, here are their docs : )"

@karlcow

This comment has been minimized.

Show comment
Hide comment
@karlcow

karlcow Sep 18, 2018

MultiMarkdown metadata? Or maybe I misunderstood the initial requirement.

karlcow commented Sep 18, 2018

MultiMarkdown metadata? Or maybe I misunderstood the initial requirement.

@scripting

This comment has been minimized.

Show comment
Hide comment
@scripting

scripting Sep 18, 2018

Owner

I'm so glad I asked this question!

It's really gratifying to see that all CMSes evolved to solve this problem.

There's a natural flow to the design of these things.

It becomes even more interesting when the content lives inside an object database.

And here's the curious thing -- GitHub repos are basically object databases. ;-)

Very much like the structures we use in Frontier to form the storage system for the website framework we developed in Early Web Days.

Owner

scripting commented Sep 18, 2018

I'm so glad I asked this question!

It's really gratifying to see that all CMSes evolved to solve this problem.

There's a natural flow to the design of these things.

It becomes even more interesting when the content lives inside an object database.

And here's the curious thing -- GitHub repos are basically object databases. ;-)

Very much like the structures we use in Frontier to form the storage system for the website framework we developed in Early Web Days.

@emk

This comment has been minimized.

Show comment
Hide comment
@emk

emk Sep 18, 2018

I've used Jekyll's YAML metadata before. It's pretty reasonable.

Pros:

  1. If a Markdown file contains Jekyll metadata, GitHub will render the metadata as a nice-looking table at the top of the file.
  2. GitHub Pages supports Jekyll, which makes it easy to publish a directory of static files as a website.
  3. Several other static site rendering tools support Jekyll metadata, so it's become a bit of a de facto standard.
  4. If you want to support lists in your metadata, you can write something like:
tags:
- programming
- thoughts

This will become ["programming", "thoughts"].

Cons:

  1. YAML looks easy to parse, but it's actually fairly complex. There are JavaScript libraries which will do all the work.
  2. Because YAML is relatively complex, YAML parsers have historically had a fair number of security bugs. This really only matters if you allow other people to upload YAML to server.

emk commented Sep 18, 2018

I've used Jekyll's YAML metadata before. It's pretty reasonable.

Pros:

  1. If a Markdown file contains Jekyll metadata, GitHub will render the metadata as a nice-looking table at the top of the file.
  2. GitHub Pages supports Jekyll, which makes it easy to publish a directory of static files as a website.
  3. Several other static site rendering tools support Jekyll metadata, so it's become a bit of a de facto standard.
  4. If you want to support lists in your metadata, you can write something like:
tags:
- programming
- thoughts

This will become ["programming", "thoughts"].

Cons:

  1. YAML looks easy to parse, but it's actually fairly complex. There are JavaScript libraries which will do all the work.
  2. Because YAML is relatively complex, YAML parsers have historically had a fair number of security bugs. This really only matters if you allow other people to upload YAML to server.
@rahuldave

This comment has been minimized.

Show comment
Hide comment
@rahuldave

rahuldave Sep 19, 2018

Pelican in Python also uses YAML frontmatter. Since jekyll supports it and provides a way for templates to get at the frontmatter, it can be used for many things. For example, I use it to support data downloads for by bayesian stats class. See http://am207.info/homeworks/AM207_HW1.html : the data download is supported in this fashion. Here is the markdown: https://github.com/AM207/2018fall/blob/master/homeworks/AM207_HW1.md

Here the markdown is produced by a custom template from a Jupyter notebook. When i need direct markdown support, I use a markdown editor such as Typora which has explicit YAML support...

rahuldave commented Sep 19, 2018

Pelican in Python also uses YAML frontmatter. Since jekyll supports it and provides a way for templates to get at the frontmatter, it can be used for many things. For example, I use it to support data downloads for by bayesian stats class. See http://am207.info/homeworks/AM207_HW1.html : the data download is supported in this fashion. Here is the markdown: https://github.com/AM207/2018fall/blob/master/homeworks/AM207_HW1.md

Here the markdown is produced by a custom template from a Jupyter notebook. When i need direct markdown support, I use a markdown editor such as Typora which has explicit YAML support...

@kwebble

This comment has been minimized.

Show comment
Hide comment
@kwebble

kwebble Sep 20, 2018

In a personal tool I've implemented this similar to HTTP headers, by separating metadata from content by 2 newlines:

key1: value1
another-key: the second value

The content of the page.

The tool knows about some keys, for example labels, and adds them in a consistent way when rendering the page.

kwebble commented Sep 20, 2018

In a personal tool I've implemented this similar to HTTP headers, by separating metadata from content by 2 newlines:

key1: value1
another-key: the second value

The content of the page.

The tool knows about some keys, for example labels, and adds them in a consistent way when rendering the page.

@scripting

This comment has been minimized.

Show comment
Hide comment
@scripting

scripting Sep 21, 2018

Owner

Question about YAML and the format for "frontmatter" in Jekyll in GitHub.

The frontmatter is delimited by a pair of ---'s.

Can I assume the line delimiter is a \n?

In other words can I search for "---\n"?

Or do I have to allow for the possibility of a \r or a combination of \r\n?

I'm only concerned with how GitHub does this.

Owner

scripting commented Sep 21, 2018

Question about YAML and the format for "frontmatter" in Jekyll in GitHub.

The frontmatter is delimited by a pair of ---'s.

Can I assume the line delimiter is a \n?

In other words can I search for "---\n"?

Or do I have to allow for the possibility of a \r or a combination of \r\n?

I'm only concerned with how GitHub does this.

@bomberstudios

This comment has been minimized.

Show comment
Hide comment
@scripting

This comment has been minimized.

Show comment
Hide comment
@scripting

scripting Sep 22, 2018

Owner

I've got YAML working with my GitHub as CMS experiment.

Here's an example of a post so you can see what it looks like.

This is exactly equivalent to using JSON, my server converts back and forth between YAML and JSON, so my app only ever sees the JSON.

BTW this post is a copy of a post I wrote in 1999 about Edit This Page functionality which was then working in Manila, our web CMS.

Owner

scripting commented Sep 22, 2018

I've got YAML working with my GitHub as CMS experiment.

Here's an example of a post so you can see what it looks like.

This is exactly equivalent to using JSON, my server converts back and forth between YAML and JSON, so my app only ever sees the JSON.

BTW this post is a copy of a post I wrote in 1999 about Edit This Page functionality which was then working in Manila, our web CMS.

@scripting

This comment has been minimized.

Show comment
Hide comment
@scripting

scripting Sep 22, 2018

Owner

JavaScript code that converts between YAML and JSON and vice versa using js-yaml.

Owner

scripting commented Sep 22, 2018

JavaScript code that converts between YAML and JSON and vice versa using js-yaml.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment