Suggestion: Read metadata in separate files. #1082

Closed
sequitur opened this Issue May 11, 2013 · 35 comments

Comments

Projects
None yet
@sequitur

Hakyll currently reads metadata from separate files, i.e. a file called foo.md would be stapled to the YAML front matter of foo.md.metadata in the same directory.

Having this in Jekyll would be helpful both for projects building their content in more formats than one (E.g., HTML via Jekyll and PDF via Pandoc). It also makes it possible to attach metadata to non-text files, which may or may not be useful at present, but would be helpful for future features or plugins.

@parkr

This comment has been minimized.

Show comment Hide comment
@parkr

parkr May 11, 2013

Member

This is a very interesting proposition. The way we avoid this at present is by placing all metadata in the YAML Front-Matter. What advantage does this have over the simplicity and extensibility of the front-matter?

Member

parkr commented May 11, 2013

This is a very interesting proposition. The way we avoid this at present is by placing all metadata in the YAML Front-Matter. What advantage does this have over the simplicity and extensibility of the front-matter?

@sequitur

This comment has been minimized.

Show comment Hide comment
@sequitur

sequitur May 11, 2013

It wouldn't replace the normal way of placing front matter, it would extend it; if the file has front matter, it would use that; if there's a separate .metadata file, Jekyll would read that and attach that metadata to the file; if there's neither, it would do what it currently does with metdata-less files (I.e., nothing).

The advantage is that it allows files to be sourced by both Jekyll and other systems with incompatible metadata. Currently, if you run a Jekyll .md file through Pandoc, Pandoc will read the YAML front matter as a weirdly formatted paragraph in between horizontal rules, and potentially mangle the start of the file. So currently, while the YAML frontmatter is simple, it doesn't play well with others - there's no easy way of separating the Jekyll-specific information from the content that might be useful to another system.

This would theoretically also lay the groundwork for Jekyll or Jekyll plugins to change how it handles files other than .md. For example, you could attach metadata to css files to programatically generate which css files are sourced by a particular html file with a tag plugin, or attach a caption as metadata to images and have a plugin that generates captioned image divs, &c.

The suggestion doesn't strictly include this, but if done in the right way, the feature could allow for multiple files to read the same metadata - e.g., a _posts.metadata file could merge its metadata to all files in _posts. With the basic metadata variables that Jekyll looks for, this isn't very useful; but when you have a more complicated set of metadata determining how content is generated, it might be a convenience feature.

It wouldn't replace the normal way of placing front matter, it would extend it; if the file has front matter, it would use that; if there's a separate .metadata file, Jekyll would read that and attach that metadata to the file; if there's neither, it would do what it currently does with metdata-less files (I.e., nothing).

The advantage is that it allows files to be sourced by both Jekyll and other systems with incompatible metadata. Currently, if you run a Jekyll .md file through Pandoc, Pandoc will read the YAML front matter as a weirdly formatted paragraph in between horizontal rules, and potentially mangle the start of the file. So currently, while the YAML frontmatter is simple, it doesn't play well with others - there's no easy way of separating the Jekyll-specific information from the content that might be useful to another system.

This would theoretically also lay the groundwork for Jekyll or Jekyll plugins to change how it handles files other than .md. For example, you could attach metadata to css files to programatically generate which css files are sourced by a particular html file with a tag plugin, or attach a caption as metadata to images and have a plugin that generates captioned image divs, &c.

The suggestion doesn't strictly include this, but if done in the right way, the feature could allow for multiple files to read the same metadata - e.g., a _posts.metadata file could merge its metadata to all files in _posts. With the basic metadata variables that Jekyll looks for, this isn't very useful; but when you have a more complicated set of metadata determining how content is generated, it might be a convenience feature.

@zachgersh

This comment has been minimized.

Show comment Hide comment
@zachgersh

zachgersh May 12, 2013

Contributor

I like the idea but it would definitely be a 2.0 change. The only thing that feels slightly odd is that you now have to define a metadata file for each file you want to parse or have a global config for all metadata. Both of those feel somewhat inferior to just having the front matter in the file.

Maybe one option is to actually have the ability to support this with a flag (like separate metadata) whereas the default behavior would always be to have the metadata within the file. If users wanted the metadata to load in separately they would add a value t the config for something like separate_metadata and that would then look in the same directory for the metadata.

Would love to see what @mattr- thinks of this.

Contributor

zachgersh commented May 12, 2013

I like the idea but it would definitely be a 2.0 change. The only thing that feels slightly odd is that you now have to define a metadata file for each file you want to parse or have a global config for all metadata. Both of those feel somewhat inferior to just having the front matter in the file.

Maybe one option is to actually have the ability to support this with a flag (like separate metadata) whereas the default behavior would always be to have the metadata within the file. If users wanted the metadata to load in separately they would add a value t the config for something like separate_metadata and that would then look in the same directory for the metadata.

Would love to see what @mattr- thinks of this.

@mattr-

This comment has been minimized.

Show comment Hide comment
@mattr-

mattr- May 12, 2013

Member

This is a well thought out idea and I can see several advantages to adopting it or making it something that is flipped by a flag in the config file. Those have already been covered well and don't need rehashing by me. They 🤘

The thing that I'm struggling with though is — when the feature is enabled — how it keeps the use of Jekyll simple for the user. Jekyll being simple for the user is, in my mind at least, based on answering 'yes' to the following questions:

  • Is it simple to run?
  • Is the metadata easy to work with when it's needed?
  • Is it simple enough to get content that doesn't require text transformation into the site?
  • Is working with Jekyll metadata, such as front matter or configuration files, obvious or is it obfuscated?

In the case of Jekyll and with the description provided, my feeling is that this is slightly crossing the line of "simple, blog-aware, static site generator". My main goal with Jekyll is to enable the simple use cases as much as possible while still keeping the hard ones possible. Considering the problem — Jekyll's metadata is incompatible with other tools — could be solved separately from Jekyll itself, I think I'm going to need a bit more information before I am comfortable with including this idea in Jekyll. For example, if I wanted to run something through Pandoc, I could totally see a user running something like remove_jekyll_metadata < foo.md | pandoc and have everything go just fine for them.

I'm not trying to play devil's advocate here, even though it may come across that way. I feel like I lack some context or some other set of insights (could just be lack of smarts too! 😉) that keeps me from jumping on board immediately. I want to read about how you'll put my fears put to rest here so I can jump on board with this feature. Thanks!

Member

mattr- commented May 12, 2013

This is a well thought out idea and I can see several advantages to adopting it or making it something that is flipped by a flag in the config file. Those have already been covered well and don't need rehashing by me. They 🤘

The thing that I'm struggling with though is — when the feature is enabled — how it keeps the use of Jekyll simple for the user. Jekyll being simple for the user is, in my mind at least, based on answering 'yes' to the following questions:

  • Is it simple to run?
  • Is the metadata easy to work with when it's needed?
  • Is it simple enough to get content that doesn't require text transformation into the site?
  • Is working with Jekyll metadata, such as front matter or configuration files, obvious or is it obfuscated?

In the case of Jekyll and with the description provided, my feeling is that this is slightly crossing the line of "simple, blog-aware, static site generator". My main goal with Jekyll is to enable the simple use cases as much as possible while still keeping the hard ones possible. Considering the problem — Jekyll's metadata is incompatible with other tools — could be solved separately from Jekyll itself, I think I'm going to need a bit more information before I am comfortable with including this idea in Jekyll. For example, if I wanted to run something through Pandoc, I could totally see a user running something like remove_jekyll_metadata < foo.md | pandoc and have everything go just fine for them.

I'm not trying to play devil's advocate here, even though it may come across that way. I feel like I lack some context or some other set of insights (could just be lack of smarts too! 😉) that keeps me from jumping on board immediately. I want to read about how you'll put my fears put to rest here so I can jump on board with this feature. Thanks!

@sequitur

This comment has been minimized.

Show comment Hide comment
@sequitur

sequitur May 12, 2013

I don't view this as an explicit configuration option - the behaviour for sites that already use Jekyll wouldn't change, at all. The change in behaviour is just that, where Jekyll currently looks in one place for metadata (The head of the file itself), it would instead look in two places and merge them (The head of the file itself, and a separate file called file.md.metadata or such). The metadata on the file itself would take precedence. So the simpler use case is still just as simple, while the more complicated use case (Jekyll being used alongside some other tool) is simpler... from the point of view of the user, anyway. How much complexity this adds depends on how Jekyll is organized internally, which is something the people who've been building Jekyll are of course better equipped to answer. 😃

Expanding this to allow for metadata that applies to multiple files, non-text files having metadata, &c., might be a future change that is enabled by this, but the key word here is future.

I don't view this as an explicit configuration option - the behaviour for sites that already use Jekyll wouldn't change, at all. The change in behaviour is just that, where Jekyll currently looks in one place for metadata (The head of the file itself), it would instead look in two places and merge them (The head of the file itself, and a separate file called file.md.metadata or such). The metadata on the file itself would take precedence. So the simpler use case is still just as simple, while the more complicated use case (Jekyll being used alongside some other tool) is simpler... from the point of view of the user, anyway. How much complexity this adds depends on how Jekyll is organized internally, which is something the people who've been building Jekyll are of course better equipped to answer. 😃

Expanding this to allow for metadata that applies to multiple files, non-text files having metadata, &c., might be a future change that is enabled by this, but the key word here is future.

@penibelst

This comment has been minimized.

Show comment Hide comment
@penibelst

penibelst May 12, 2014

Member

Use the data feature instead. @jekyll/owners: can be closed.

Member

penibelst commented May 12, 2014

Use the data feature instead. @jekyll/owners: can be closed.

@parkr

This comment has been minimized.

Show comment Hide comment
@parkr

parkr May 12, 2014

Member

This has been requested by a bunch of folks.

Member

parkr commented May 12, 2014

This has been requested by a bunch of folks.

@mscharley

This comment has been minimized.

Show comment Hide comment
@mscharley

mscharley May 12, 2014

Contributor

It only gets worse with collections. There was another issue that would
have been solved perfectly with collections except his files were PDF... No
way to include YAML. The same is true if I want a collection of images to
represent a sideshow.
On 12/05/2014 3:27 pm, "Parker Moore" notifications@github.com wrote:

This has been requested by a bunch of folks.


Reply to this email directly or view it on GitHubhttps://github.com/jekyll/jekyll/issues/1082#issuecomment-42797117
.

Contributor

mscharley commented May 12, 2014

It only gets worse with collections. There was another issue that would
have been solved perfectly with collections except his files were PDF... No
way to include YAML. The same is true if I want a collection of images to
represent a sideshow.
On 12/05/2014 3:27 pm, "Parker Moore" notifications@github.com wrote:

This has been requested by a bunch of folks.


Reply to this email directly or view it on GitHubhttps://github.com/jekyll/jekyll/issues/1082#issuecomment-42797117
.

@penibelst

This comment has been minimized.

Show comment Hide comment
@penibelst

penibelst May 12, 2014

Member

Data files are separate files. The design pattern is to create _data/assets.yaml:

# id: 0
-
  url: /assets/photo.jpg
  type: image/jpeg
  name: My photo
  width: 300
  height: 150

#

# id: 17
-
  url: /assets/book.pdf
  type: application/pdf
  name: My book

Then you can access the book entry at the id with site.data.assets[17]. You can create includes with parameters for every type you want to embed. For example:

<!-- _includes/image.html (id, class)-->
{% assign id = include.id | times: 1 %}
{% assign image = site.data.images[id] %}
<img
  class="{{ include.class }}"
  alt="{{ image.name }}"
  src="{{ image.url }}"
/>

I use this pattern since the data feature is out.

Member

penibelst commented May 12, 2014

Data files are separate files. The design pattern is to create _data/assets.yaml:

# id: 0
-
  url: /assets/photo.jpg
  type: image/jpeg
  name: My photo
  width: 300
  height: 150

#

# id: 17
-
  url: /assets/book.pdf
  type: application/pdf
  name: My book

Then you can access the book entry at the id with site.data.assets[17]. You can create includes with parameters for every type you want to embed. For example:

<!-- _includes/image.html (id, class)-->
{% assign id = include.id | times: 1 %}
{% assign image = site.data.images[id] %}
<img
  class="{{ include.class }}"
  alt="{{ image.name }}"
  src="{{ image.url }}"
/>

I use this pattern since the data feature is out.

@penibelst

This comment has been minimized.

Show comment Hide comment
@penibelst

penibelst May 12, 2014

Member

Of course you can apply the new filters like where or group_by to the data.

Member

penibelst commented May 12, 2014

Of course you can apply the new filters like where or group_by to the data.

@archagon

This comment has been minimized.

Show comment Hide comment
@archagon

archagon Jun 24, 2014

A vote for this feature. To me, Jekyll is a way of converting my pristine, human-readable and editable *.md files into messy, blog-ready HTML. I think many other users of static blogging engines see it the same way: many of my favorite bloggers prefer to do their writing in clean, distraction-free apps and use Markdown precisely for the simplicity of the syntax. Unfortunately, the YAML metadata not only looks ugly, but it messes up the formatting in most Markdown editors. For my own personal use, I'm going to write a preprocessing script or plugin that looks in the _posts directory and merges any *.yaml files with the same name as an *.md file, but it would be nice to have this as a built-in feature!

A vote for this feature. To me, Jekyll is a way of converting my pristine, human-readable and editable *.md files into messy, blog-ready HTML. I think many other users of static blogging engines see it the same way: many of my favorite bloggers prefer to do their writing in clean, distraction-free apps and use Markdown precisely for the simplicity of the syntax. Unfortunately, the YAML metadata not only looks ugly, but it messes up the formatting in most Markdown editors. For my own personal use, I'm going to write a preprocessing script or plugin that looks in the _posts directory and merges any *.yaml files with the same name as an *.md file, but it would be nice to have this as a built-in feature!

@troyswanson

This comment has been minimized.

Show comment Hide comment
@troyswanson

troyswanson Jun 24, 2014

Member

@archagon Which Markdown editors does YAML Front-matter not play nice with? I've never had a problem; the editors I've used mostly just ignore the Front-matter since it doesn't follow MD syntax...

Member

troyswanson commented Jun 24, 2014

@archagon Which Markdown editors does YAML Front-matter not play nice with? I've never had a problem; the editors I've used mostly just ignore the Front-matter since it doesn't follow MD syntax...

@archagon

This comment has been minimized.

Show comment Hide comment
@archagon

archagon Jun 24, 2014

Here's how Mou renders the Markdown in the left pane:

Does it matter? Maybe not to some people, but to me, it's really distracting and makes me lose focus. It also kind of sullies the pristine-ness of my *.md files. I'd much rather have that metadata separate.

Here's how Mou renders the Markdown in the left pane:

Does it matter? Maybe not to some people, but to me, it's really distracting and makes me lose focus. It also kind of sullies the pristine-ness of my *.md files. I'd much rather have that metadata separate.

@troyswanson

This comment has been minimized.

Show comment Hide comment
@troyswanson

troyswanson Jun 24, 2014

Member

I totally understand about the focus issue. My workflow consists of opening a new document in iA Writer, writing my post, and then copying/pasting that content into a new file in my Jekyll repo. This way I'm able to separate the content creation piece (writing the post) from the administrative piece (publishing the post).

Does it matter?

Yes. For better or for worse, every change that is proposed to a wildly popular open-source project like Jekyll demands rigorous debate, especially considering one of the main goals of Jekyll is to be as obvious in the way it does things as possible.

If there are multiple ways to do something as simple as specifying post metadata, then that makes it harder for the community to support people who are having trouble with stuff, since there are more variables to deal with.

Also, it increases the learning curve of the project, increases the amount of documentation that needs to be written, increases the number of tests that need to run... All of that said, the main point is just that the community needs to decide if the tradeoff for the new functionality described here is worth all of that extra stuff I listed.

Member

troyswanson commented Jun 24, 2014

I totally understand about the focus issue. My workflow consists of opening a new document in iA Writer, writing my post, and then copying/pasting that content into a new file in my Jekyll repo. This way I'm able to separate the content creation piece (writing the post) from the administrative piece (publishing the post).

Does it matter?

Yes. For better or for worse, every change that is proposed to a wildly popular open-source project like Jekyll demands rigorous debate, especially considering one of the main goals of Jekyll is to be as obvious in the way it does things as possible.

If there are multiple ways to do something as simple as specifying post metadata, then that makes it harder for the community to support people who are having trouble with stuff, since there are more variables to deal with.

Also, it increases the learning curve of the project, increases the amount of documentation that needs to be written, increases the number of tests that need to run... All of that said, the main point is just that the community needs to decide if the tradeoff for the new functionality described here is worth all of that extra stuff I listed.

@archagon

This comment has been minimized.

Show comment Hide comment
@archagon

archagon Jun 24, 2014

Sorry! That "Does it matter?" wasn't aimed at you. I meant it in the rhetorical sense, as in, "Does this metadata issue matter to most ordinary folks? Probably not, but it bothers me." I of course understand why you are asking for clarification, and why rigorous debate must be involved to even consider this feature! :)

"My workflow consists of opening a new document in iA Writer, writing my post, and then copying/pasting that content into a new file in my Jekyll repo."

One of the main reasons I decided to try Jekyll after dealing with Wordpress for so long is that I wanted to avoid this sort of copy-and-paste blogging. My ideal system has a single source of truth — the Markdown files — which are then directly used to generate the blog. Many other bloggers I follow use the same workflow, only they've resorted to writing their own blogging engines because the current ones don't fit the bill for whatever reason. Personally, I think this feature would make Jekyll a lot nicer to use for that particular breed of picky blogger (including myself). Plus, it's easy to describe and ordinary users wouldn't have to deal with it at all.

Sorry! That "Does it matter?" wasn't aimed at you. I meant it in the rhetorical sense, as in, "Does this metadata issue matter to most ordinary folks? Probably not, but it bothers me." I of course understand why you are asking for clarification, and why rigorous debate must be involved to even consider this feature! :)

"My workflow consists of opening a new document in iA Writer, writing my post, and then copying/pasting that content into a new file in my Jekyll repo."

One of the main reasons I decided to try Jekyll after dealing with Wordpress for so long is that I wanted to avoid this sort of copy-and-paste blogging. My ideal system has a single source of truth — the Markdown files — which are then directly used to generate the blog. Many other bloggers I follow use the same workflow, only they've resorted to writing their own blogging engines because the current ones don't fit the bill for whatever reason. Personally, I think this feature would make Jekyll a lot nicer to use for that particular breed of picky blogger (including myself). Plus, it's easy to describe and ordinary users wouldn't have to deal with it at all.

@penibelst

This comment has been minimized.

Show comment Hide comment
@penibelst

penibelst Jun 24, 2014

Member

@archagon Do you know about front-matter defaults? They tackle your issues.

Member

penibelst commented Jun 24, 2014

@archagon Do you know about front-matter defaults? They tackle your issues.

@archagon

This comment has been minimized.

Show comment Hide comment
@archagon

archagon Jun 24, 2014

Wouldn't you still have to have title/date/category YAML for most blog posts?

Wouldn't you still have to have title/date/category YAML for most blog posts?

@troyswanson

This comment has been minimized.

Show comment Hide comment
@troyswanson

troyswanson Jun 24, 2014

Member

Sorry! That "Does it matter?" wasn't aimed at you. I meant it in the rhetorical sense, as in, "Does this metadata issue matter to most ordinary folks? Probably not, but it bothers me".

Ah! Glad that clarification was made; I almost went into rage-mode. :rage2:


My ideal system has a single source of truth — the Markdown files — which are then directly used to generate the blog.

I can definitely appreciate this. I'm trying to think of a way to do what you're proposing, but with features that exist right now. Unfortunately, as far as I know, the only way Jekyll will actually parse a Markdown file is if the file contains those two lines of three dashes:

---
categories: gaming
---

# a blog post

Do you know about front-matter defaults? They tackle your issues.

@penibelst I think you'd still need to have those two lines of three dashes. This is what is causing the main rendering problems in Mou in the first place, and of course removes the pristine Markdown file stuff.

screen shot 2014-06-24 at 3 28 59 pm


An entirely different angle: maintaining two files side-by-side cannot be fun. For instance, something like my-awesome-blog-post.md and my-awesome-blog-post.md.metadata seems (to me at least) a very inelegant solution to this problem. I like the idea of using data files, but you'd still have to maintain a list of many-to-one blog posts in there.

@parkr I'd love to hear your take on this. What do you think?

Member

troyswanson commented Jun 24, 2014

Sorry! That "Does it matter?" wasn't aimed at you. I meant it in the rhetorical sense, as in, "Does this metadata issue matter to most ordinary folks? Probably not, but it bothers me".

Ah! Glad that clarification was made; I almost went into rage-mode. :rage2:


My ideal system has a single source of truth — the Markdown files — which are then directly used to generate the blog.

I can definitely appreciate this. I'm trying to think of a way to do what you're proposing, but with features that exist right now. Unfortunately, as far as I know, the only way Jekyll will actually parse a Markdown file is if the file contains those two lines of three dashes:

---
categories: gaming
---

# a blog post

Do you know about front-matter defaults? They tackle your issues.

@penibelst I think you'd still need to have those two lines of three dashes. This is what is causing the main rendering problems in Mou in the first place, and of course removes the pristine Markdown file stuff.

screen shot 2014-06-24 at 3 28 59 pm


An entirely different angle: maintaining two files side-by-side cannot be fun. For instance, something like my-awesome-blog-post.md and my-awesome-blog-post.md.metadata seems (to me at least) a very inelegant solution to this problem. I like the idea of using data files, but you'd still have to maintain a list of many-to-one blog posts in there.

@parkr I'd love to hear your take on this. What do you think?

@penibelst

This comment has been minimized.

Show comment Hide comment
@penibelst

penibelst Jun 24, 2014

Member

I think you'd still need to have those two lines of three dashes. This is what is causing the main rendering problems in Mou in the first place, and of course removes the pristine Markdown file stuff.

@troyswanson Sounds like we need better text editors. Ask the Mou author to add an option to ignore yaml metadata, or render it like Github does.

Markdown specification is too vague about headings (#, ---, ===) and other markup. This is why collisions with other languages inside one file are unavoidable.

Member

penibelst commented Jun 24, 2014

I think you'd still need to have those two lines of three dashes. This is what is causing the main rendering problems in Mou in the first place, and of course removes the pristine Markdown file stuff.

@troyswanson Sounds like we need better text editors. Ask the Mou author to add an option to ignore yaml metadata, or render it like Github does.

Markdown specification is too vague about headings (#, ---, ===) and other markup. This is why collisions with other languages inside one file are unavoidable.

@archagon

This comment has been minimized.

Show comment Hide comment
@archagon

archagon Jun 24, 2014

Well, ultimately, if a Markdown file contains a YAML header, then it's not really a Markdown file anymore, is it? It's a weird YAML-Markdown hybrid. I don't think it's fair to ask text editor authors to deviate from the Markdown specification just to support this (from their perspective) arbitrary mixture of formats.

Well, ultimately, if a Markdown file contains a YAML header, then it's not really a Markdown file anymore, is it? It's a weird YAML-Markdown hybrid. I don't think it's fair to ask text editor authors to deviate from the Markdown specification just to support this (from their perspective) arbitrary mixture of formats.

@parkr

This comment has been minimized.

Show comment Hide comment
@parkr

parkr Jul 31, 2014

Member

I don't dislike this idea at all anymore. it adds complexity, but it's your choice to live with the complexity.

Let's wait on Jekyll 3.0 for this one tho.

Member

parkr commented Jul 31, 2014

I don't dislike this idea at all anymore. it adds complexity, but it's your choice to live with the complexity.

Let's wait on Jekyll 3.0 for this one tho.

@wanzhou

This comment has been minimized.

Show comment Hide comment
@wanzhou

wanzhou Aug 4, 2014

About data files, whether all the parameters I want need to input by hand? If I can do it by *.rb?

wanzhou commented Aug 4, 2014

About data files, whether all the parameters I want need to input by hand? If I can do it by *.rb?

@benbalter

This comment has been minimized.

Show comment Hide comment
@benbalter

benbalter Aug 4, 2014

Contributor

Jekyll: Where there's three ways to do everything.

Part of Jekyll's attractiveness as a publishing platform is its simplicity. Need some metadata? Great. YAML frontmatter. Every time we add a "or you could also use this alternate method" to the docs (see markdown interpreters, highlighters, etc.) we make it harder for new users to learn the platform.

It looks like the discussion strayed a bit into how some editors render frontmatter (FWIW, just add an extra line break after the last key/value pair), but at it's core, I only see one use case (using Jekyll content in PanDoc), which seems like a non-dominant workflow. Do the majority of our users need this feature?

FWIW, if it were me, if I absolutely couldn't have front matter in a file, I'd have foo.yml in the _data folder, and access by common slug.

As a last note, YAML frontmatter is how we distinguish Jekyll files from static HTML files (e.g., processing Liquid + converters). Removing that flag (or potentially adding a second possible flag) adds yet another layer of complexity.

Contributor

benbalter commented Aug 4, 2014

Jekyll: Where there's three ways to do everything.

Part of Jekyll's attractiveness as a publishing platform is its simplicity. Need some metadata? Great. YAML frontmatter. Every time we add a "or you could also use this alternate method" to the docs (see markdown interpreters, highlighters, etc.) we make it harder for new users to learn the platform.

It looks like the discussion strayed a bit into how some editors render frontmatter (FWIW, just add an extra line break after the last key/value pair), but at it's core, I only see one use case (using Jekyll content in PanDoc), which seems like a non-dominant workflow. Do the majority of our users need this feature?

FWIW, if it were me, if I absolutely couldn't have front matter in a file, I'd have foo.yml in the _data folder, and access by common slug.

As a last note, YAML frontmatter is how we distinguish Jekyll files from static HTML files (e.g., processing Liquid + converters). Removing that flag (or potentially adding a second possible flag) adds yet another layer of complexity.

@parkr

This comment has been minimized.

Show comment Hide comment
@parkr

parkr Aug 5, 2014

Member

The more flexibility we offer, the more we can enable power users.

There are already several ways to do a bunch of things – we like options. Curiously, whenever we only allow one way to do something (such as pagination), we get chewed out for being rigid and stupid for not implementing someone else's idea of how pagination should work. I'm happy to live with being chewed out, but to some degree, offering flexibility makes sense. This is where sane defaults come in.

Editors aren't updating to render front matter properly (except vim, but that's because it's the very best text editor), or there is a performance slump if the feature is added. We only have two items here, and one of them is mine: http://jekyllrb.com/docs/plugins/#editors

At any rate, a plugin could easily be written to monkey patch Convertible#read_yaml and Site#has_yaml_header? to use this new functionality, so it's not as though we have to decide this right now. Can a pandoc plugin be written to strip out the YAML front matter?

Member

parkr commented Aug 5, 2014

The more flexibility we offer, the more we can enable power users.

There are already several ways to do a bunch of things – we like options. Curiously, whenever we only allow one way to do something (such as pagination), we get chewed out for being rigid and stupid for not implementing someone else's idea of how pagination should work. I'm happy to live with being chewed out, but to some degree, offering flexibility makes sense. This is where sane defaults come in.

Editors aren't updating to render front matter properly (except vim, but that's because it's the very best text editor), or there is a performance slump if the feature is added. We only have two items here, and one of them is mine: http://jekyllrb.com/docs/plugins/#editors

At any rate, a plugin could easily be written to monkey patch Convertible#read_yaml and Site#has_yaml_header? to use this new functionality, so it's not as though we have to decide this right now. Can a pandoc plugin be written to strip out the YAML front matter?

@benbalter

This comment has been minimized.

Show comment Hide comment
@benbalter

benbalter Aug 5, 2014

Contributor

The more flexibility we offer, the more we can enable power users.

Until a point. See the Microsoft Word Ribbon. Users need software to make choices to keep it usable.

a plugin could easily be written

I'd ❤️ to see features like this begin their life as a third-party plugin... experiment on the workflow a bit and tweak, become a "core" plugin, and maybe then, eventually make it into core. Would allow us to get feedback on potentially cool features like this, without the need to go all in or nothing.

Contributor

benbalter commented Aug 5, 2014

The more flexibility we offer, the more we can enable power users.

Until a point. See the Microsoft Word Ribbon. Users need software to make choices to keep it usable.

a plugin could easily be written

I'd ❤️ to see features like this begin their life as a third-party plugin... experiment on the workflow a bit and tweak, become a "core" plugin, and maybe then, eventually make it into core. Would allow us to get feedback on potentially cool features like this, without the need to go all in or nothing.

@parkr

This comment has been minimized.

Show comment Hide comment
@parkr

parkr Aug 5, 2014

Member

I'd ❤️ to see features like this begin their life as a third-party plugin... experiment on the workflow a bit and tweak, become a "core" plugin, and maybe then, eventually make it into core. Would allow us to get feedback on potentially cool features like this, without the need to go all in or nothing.

👍 to this philosophy.

Ping me in an issue if you need help!

Member

parkr commented Aug 5, 2014

I'd ❤️ to see features like this begin their life as a third-party plugin... experiment on the workflow a bit and tweak, become a "core" plugin, and maybe then, eventually make it into core. Would allow us to get feedback on potentially cool features like this, without the need to go all in or nothing.

👍 to this philosophy.

Ping me in an issue if you need help!

@Ajedi32

This comment has been minimized.

Show comment Hide comment
@Ajedi32

Ajedi32 May 5, 2015

👍 for this. Unless I'm misunderstanding here, it's currently impossible to use standard markdown-formatted files with Jekyll and have them render (same for other file formats too). Unless you include weird, Jekyll-specific metadata at the top of all your files, Jekyll refuses to process them. That's kind of a deal breaker for me at the moment.

Ajedi32 commented May 5, 2015

👍 for this. Unless I'm misunderstanding here, it's currently impossible to use standard markdown-formatted files with Jekyll and have them render (same for other file formats too). Unless you include weird, Jekyll-specific metadata at the top of all your files, Jekyll refuses to process them. That's kind of a deal breaker for me at the moment.

@micapam

This comment has been minimized.

Show comment Hide comment
@micapam

micapam Jun 14, 2015

Me too, @Ajedi32. I'm going to use nanoc for now, but I'd rather use Jekyll if possible as there seems to be wider community support etc. So I'll watch this thread.

My thoughts on the 'simplicity' comments above: there is more than one kind of simplicity. I like Markdown because it's minimal, and system-agnostic. Simple for the user != simple for the system.

micapam commented Jun 14, 2015

Me too, @Ajedi32. I'm going to use nanoc for now, but I'd rather use Jekyll if possible as there seems to be wider community support etc. So I'll watch this thread.

My thoughts on the 'simplicity' comments above: there is more than one kind of simplicity. I like Markdown because it's minimal, and system-agnostic. Simple for the user != simple for the system.

@parkr

This comment has been minimized.

Show comment Hide comment
@parkr

parkr Aug 26, 2015

Member

Are any of you using a plugin for this? What does it look like? I think we'll ship Jekyll 3 without this and revisit later. I have not heard anything for a while so it's becoming lower priority.

Member

parkr commented Aug 26, 2015

Are any of you using a plugin for this? What does it look like? I think we'll ship Jekyll 3 without this and revisit later. I have not heard anything for a while so it's becoming lower priority.

@Ajedi32

This comment has been minimized.

Show comment Hide comment
@Ajedi32

Ajedi32 Aug 26, 2015

A plugin would be great. Personally, my interest has shifted to Metalsmith though so I don't really have the motivation to write one myself at the moment (especially given that I'm not very familiar with Jekyll or its plugin system).

If there is a plugin which implements this functionality though I'd certainly love to hear about it.

Ajedi32 commented Aug 26, 2015

A plugin would be great. Personally, my interest has shifted to Metalsmith though so I don't really have the motivation to write one myself at the moment (especially given that I'm not very familiar with Jekyll or its plugin system).

If there is a plugin which implements this functionality though I'd certainly love to hear about it.

@micapam

This comment has been minimized.

Show comment Hide comment
@micapam

micapam Aug 26, 2015

@parkr Beware self-selection bias. You not be hearing much about it because those people for whom it's a deal-breaker are going elsewhere. I considered Jekyll but decided on nanoc for this very reason.

micapam commented Aug 26, 2015

@parkr Beware self-selection bias. You not be hearing much about it because those people for whom it's a deal-breaker are going elsewhere. I considered Jekyll but decided on nanoc for this very reason.

@jooize

This comment has been minimized.

Show comment Hide comment
@jooize

jooize Aug 27, 2015

I'm using Hardwired much because of its support for omitting the YAML “---” and allowing capital letters in the front matter tags “Layout: legal-page”.

jooize commented Aug 27, 2015

I'm using Hardwired much because of its support for omitting the YAML “---” and allowing capital letters in the front matter tags “Layout: legal-page”.

@envygeeks

This comment has been minimized.

Show comment Hide comment
@envygeeks

envygeeks Aug 27, 2015

Contributor

The latter part is a bit over the top to ask for IMO, those are keys that are meant for Jekyll and to have to lowercase them just because you want to uppercase them when we expected them to be lowercase is a step backwards in performance one that grows along with other issues as the site grows. Those aren't problems that most people are concerned with but simply put: it's a configuration option.

Contributor

envygeeks commented Aug 27, 2015

The latter part is a bit over the top to ask for IMO, those are keys that are meant for Jekyll and to have to lowercase them just because you want to uppercase them when we expected them to be lowercase is a step backwards in performance one that grows along with other issues as the site grows. Those aren't problems that most people are concerned with but simply put: it's a configuration option.

@karelv

This comment has been minimized.

Show comment Hide comment
@karelv

karelv Jan 30, 2016

👍 to the original suggestion (and keeping the front matter as is, and thus mandatory)

karelv commented Jan 30, 2016

👍 to the original suggestion (and keeping the front matter as is, and thus mandatory)

@karelv

This comment has been minimized.

Show comment Hide comment
@karelv

karelv Jan 30, 2016

Made a generator plugin. Probably not 100% optimized or correct, but it does do what I need. (new on ruby and jekyll)

module ReadMetaFiles
  class Generator < Jekyll::Generator
    def generate(site)

      site.posts.docs.each do |doc|
        meta_file = (doc.path + ".yml")
        if File.exist?(meta_file)
            meta_data = SafeYAML.load_file(meta_file)
            meta_data.each { |key, value| doc.data[key] = value}
        end
      end

      site.pages.each do |doc|
        meta_file = (doc.path + ".yml")
        if File.exist?(meta_file)
            meta_data = SafeYAML.load_file(meta_file)
            meta_data.each { |key, value| doc.data[key] = value}
        end
      end

    end
  end
end

So place this rb-file in your _plugins directory. Now each post and page will try to load an additional Front Matter in the yml file name afer the post(or page) file-name with an extra .yml suffix.

Anyone an idea of the caveats?

karelv commented Jan 30, 2016

Made a generator plugin. Probably not 100% optimized or correct, but it does do what I need. (new on ruby and jekyll)

module ReadMetaFiles
  class Generator < Jekyll::Generator
    def generate(site)

      site.posts.docs.each do |doc|
        meta_file = (doc.path + ".yml")
        if File.exist?(meta_file)
            meta_data = SafeYAML.load_file(meta_file)
            meta_data.each { |key, value| doc.data[key] = value}
        end
      end

      site.pages.each do |doc|
        meta_file = (doc.path + ".yml")
        if File.exist?(meta_file)
            meta_data = SafeYAML.load_file(meta_file)
            meta_data.each { |key, value| doc.data[key] = value}
        end
      end

    end
  end
end

So place this rb-file in your _plugins directory. Now each post and page will try to load an additional Front Matter in the yml file name afer the post(or page) file-name with an extra .yml suffix.

Anyone an idea of the caveats?

@jekyllbot jekyllbot added the stale label Jun 6, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment