Markdown in .yml files (sort of) broken. A way to link externally?

[lunelson]

So, I was getting pretty stoked about the 3.x features, Twig and everything, but the switch to YML files breaks my favorite part about the markdown workflow:

with the more complicated syntax (indenting all lines of markdown content by two spaces), standalone markdown editors (Mou, Byword, Marked, etc.) don't understand it correctly, and the native previewing in those apps doesn't work anymore.

Therefore my question, is if (and how) there is any chance that either:

1. we can remove this requirement of the 2-space indent, or
2. we can simply link to .md markdown files from the .yml file, allowing the possibility to write content in a native markdown format

Thoughts?

[kvakes]

I agree that from the end user point of view it just doesn't make sense to have markdown content indented. 1 is better option though.

[kvakes]

On the other hand, it's YAML's native format for inserting large chunks of text, so, there must be a better solution to the problem.

[lunelson]

Yes, I figured that the first option might not be possible, since it's trading off violating one format's rules for another's;

that's why I wondered whether it wouldn't be possible to link the file–putting something like the following, in the page.yml file:

content: {{path}}content.md

… which would also allow you to have pretty much any name for the content file (as long as the corresponding bit exists in the YML file), and it might even be more semantic this way: since the YAML file (named correspondingly with its template) is just a collection of key/value pairs and links, and the content text and media files are separate, with names according to their own context.

[kvakes]

Since there's only one .yml file in a folder, we as well might just add content.md to this folder in order to initialize content variable that will contain Markdown. Also, we could add something like text.html with HTML content, etc. Convention over configuration, y'know.

What you think?

[lunelson]

yes, that could work, though not every page type will contain a content block called "content", nor necessarily contain only one block.

I'm considering that the key file for any page is the .yml file, and that text (i.e. non-media) content is either written in-line in that file (e.g. title: My Page), or could be linked to an extra file if necessary:

content-block-1: {{path}}content-1.md
content-block-2: {{path}}content-2.html

Note: the above syntax is most probably wrong

[kvakes]

My initial thought was than you can have as many .md files as you like and name them as you please, so in case that you have the following directory structure:

pagename.yml
content.md
long_text.md
another.md
something.html

you'll end up with context containing all the variables from pagename.yml plus variables page.content, page.long_text, page.another filled with whatever you place in corresponding files; and (maybe!) page.something with HTML content.

[lunelson]

Ah yes, I just understood what you meant before.

Indeed, that might be the most straightforward for the user, if the parser just looks for anything with an .md or .html extension and appends it to the values for that page (keys being equal to the filename). Similar to the way it collects media information. And any other key-value pairs go in to the YAML file, and the template name still corresponds to whatever the YAML file is called. That would be really powerful.

Any of the development contributors care to weigh in on this?

[kolber]

This is really interesting.
The double-space indent has been something that has been irritating me with the new version as well.

I like the evolution of the idea that @kvakes is proposing. As YAML doesn't have an external file-loading mechanic, one would have to be added, which would break with convention somewhat.

But, the idea of using the filename as the key makes a lot of sense. In principal it feels like quite an elegant solution.

[kvakes]

HI, Anthony.

Including markdown files as variables doesn't have any relation to YAML, it's just another way to init a variable in template context, so we're not breaking any conventions. And users will be able to decide whether to define long passages of text via YAML or via external files.

One problem that I can see is that in earlier version of Stacey we could have $collections of .html files, for example. Or collections of any "other than .txt" files. And our new feature is somewhat in conflict with what we had before.

[kolber]

Oh, with the 'YAML doesn't have an external file-loading mechanic' comment, I was referring to the idea of allowing
content-block-1: {{path}}content-1.md
This would mean extending the YAML parser, which I would rather avoid. Your alternative solution of including .md files as page variables is a very nice way of bypassing this issue.

I don't actually think that this will cause any conflicts with the $html collections. I think the two could sit side by side without causing problems.

[lunelson]

I see what @kvakes is getting at with the potential conflict with $collections; though maybe it's not quite a direct conflict?

In any case it suggests to me two thoughts:

1. Perhaps it would be possible to allow all files other than the .yml file in a folder (perhaps also excluding named collections like _thumbnails) to be registered as values with the filename as their key, while also still allowing them to be collected in to $collection arrays in the old way too?; or alternately
2. Maybe there is a way to call directly on the file rather than looping over collections?

Something like

{{page.file(content.md).content}}

which would assume the following is also possible

{{page.file(content.md).name}}
{{page.file(content.md).file_name}}
{{page.file(content.md).url}}

etc… I'm basing that on a combination of what I'm reading in the 2.3.x and 3.x documentation

[lunelson]

On the way home I thought about this further, and would offer one more observation:

The $collection thing makes sense for data types that are externally readable and will be passed directly in to html output via template instructions. Logically, in parsing these the system collects more than just the @content of those files, it collects also the @name, @file_name, @url, etc.

But what's unique about the usage of .yml and .md (and for that matter .txt) is that they are only parsed internally -- not inserted directly in the html output -- and the only relevant information collected from them is the @file_name and the @content (which we can assume to be UTF-8 text).

So maybe the idea of parsing page value directly from files, keying on the filename, should be restricted to these types?

I still think a syntax for referring to a file by name would be great (see my previous comment), but that's another feature request I think.

Anyway I'm really excited to think this feature might see the light… good luck

[lunelson]

Hmm. No interest in this issue from the active developers? Maybe we should change the title ;)

[kolber]

I love the idea, but I haven't had a chance to look at it yet.

[lunelson]

Cool. Glad to know it

[sarafen]

I looked around for thoughts on this indentation issue, only to find this issue after I posted this diatribe (mainly because the title wasn't very obvious at first), (#61).

In summarization it seems most of my issues are bugging others as well, but I'm not sold on the inclusion of a .md file into the .yml file as the sole solution.

My concern here (and correct me if I misread), is that with this approach your Title, Date, etc. is all in your .yml file, and your Content is in your .md file. That's a wasted separation. I think there really should be a two pronged approach to this.

1. Allow the inclusion of .md files as said above, this works great if you have loads of page content that you want to separate out into more manageable .md chunks.
2. Allow a .md wrapper of sorts to be included in the .yml file itself, that will parse anything within the wrap "tags" the same as an included .md file. This way if your content is one paragraph, but needs to be .md AND needs to avoid the annoying indentation, you don't have to create another file, just for one paragraph.

So you end up with something like this (excuse the rough syntax):

File Inclusion:

content(md) : {{path}}content.md

OR

Block Inclusion:

content(md) : |

All your content are belong to us.

If the parser gathers a key that ends with '(md)', then it looks for a file path in the value, if a pipe is found instead then it processes the following block as .md without required indentation.

For content context, I also find it especially helpful to have the Date, Title, etc. in the same file as the content. Forcing a separation, makes it harder to keep up with all your content.

I understand in keeping with YAML convention, but this is one area where YAML fails miserably in its lack of non-indented block literal handling, so its worth considering a slight "modification" here, to help maintain more honest user input -> output.

[kolber]

Totally agree with these concerns and it is something that has been causing me issues.
I have been playing around with different solutions on some of my own projects, including the external .md file approach.

I tend to think that fencing markdown content is the better approach, as it does keep all of your content together in a single file. Which makes management much easier.

This is the syntax I have been playing with:

title: The Page Title

subtitle: +++
A shorter _piece_ of markdown.
+++

content: +++
## A markdown title

And a paragraph containing **bold** text. 
+++

Where the +++s can sit on their own lines, or contain more than three +s:

title: The Page Title

subtitle:
++++++++++++++++++++++++++++++++++++
A shorter _piece_ of markdown.
++++++++++++++++++++++++++++++++++++

content:
++++++++++++++++++++++++++++++++++++

## A markdown title

And a paragraph containing **bold** text.

++++++++++++++++++++++++++++++++++++

[f0x2501]

Kolber,
Your solution rock's.
I also think that keeping all page content in one file is important.

There is also the suffix way

content.md:
content_md:
content^md:
content_md_:
...

[o-l-e]

hey i like both the last two suggestions.

@mentat-fr how would you close the tag, or would this automatically happen when writing a new variable below this?
Excuse me, i am unfamiliar with yml.

This may be a silly suggestion (from a user perspective), but if you are able to use a suffix to control wether to use markdown or not, then could you also use for example a semicolon or other character?

like

variable: This is not markdown

variable;
This is markdown etc

[kvakes]

I agree with @StephenLovell that it's not very good to store variables in different files. Two best solutions, from my point of view, so far are:

• content(.md|(md)|_md) variable syntax
• content.md type of files (without explicit path setting)

Since Stacey uses a very small subset of Yaml (?), we could just restrain Stacey from using Yaml, e.g. rollback to old Stacey variant where if a variable starts with \n then it is treated as Markdown.

If we want to stick with Yaml, I would not change the syntax, it's just wrong to change something that is maintained by others, we no longer will be able to say that it is Yaml, because 1) users will be confused, because our Yaml is different, 2) the "wrong" syntax can have other dependencies in Yaml, or just philosophy we do not understand. We could also bug Clark Evans about it. But if we want Yaml's complexity, I would stick with *.md files treated as variables. It is not perfect solution, I understand it.

[sarafen]

thus far my favorite is

title: Amazing Title Here

content.md: 
A bunch of content here

I get the desire to follow yaml syntax, but I'd sacrifice that standard over UX any day.

This bends the rules, omits the indentation requirement and the pipe character. However your key is still a key, and you can do the markdown parsing easily by scanning all keys with .md at the end.

Inluding external files is a useful option, but should not be the default.

[kolber]

Unfortunately we need to specify both the start and the end of markdown content for parsing, otherwise the solutions listed above would be great.

The basic problem is that we need to pre-parse the text file and convert the markdown blocks into YAML compatible text blocks before passing them back to the YAML parser.

Unless you guys had some alternate ideas on how we could implement the above?

[sarafen]

Forgive me for my ignorance, but I guess I was assuming that the colon indicated the beginning of a VALUE and a new line plus next VALUE or EOF indicated the ending of a VALUE. A KEY with a .md suffix marks that it should be handled as markdown.

So

content: is just a regular KEY

&

content.md: is a KEY that is flagged for markdown parsing.

In a nutshell
Any KEY with a .md suffix is parsed into a yaml block literal, and shot to yaml for parsing. And all VALUEs begin with a COLON and end with a NEWLINE + NEXT KEY or EOF.

To me that seems to omit the need to overtly flag the start and finish with +++, yes the preprocessor has to do a bit more heavy lifting, but it seems to me to be possible to incorporate the above to indicate the start and stop of VALUEs.

Or am I missing something?

[kolber]

It is a little more tricky than that.

Take the following example:

title: Page title

subtitle.md: 
What if a colon appears in an awkward position within the text block?
Like: this.

content.md:
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Vivamus luctus urna sed urna ultricies 
ac tempor dui sagittis. In condimentum facilisis porta. Sed nec diam eu diam mattis viverra. 
Nulla fringilla, orci ac euismod semper, magna diam porttitor mauris, quis sollicitudin sapien 
justo in libero. Vestibulum.

It is very tricky for us to know with absolute certainty where the textblock ends and a new key starts.
This is why YAML uses indentation to differentiate textblocks from keys.

[jaywilliams]

Have you considerd using YAML front matter style syntax, using YAML's native document syntax?
Example: https://github.com/mojombo/jekyll/wiki/YAML-Front-Matter

[kolber]

I have, and I quite like the approach, especially as it is quite common these days.

The problem is that I often have use for several blocks of markdown content — which could be handled with multiple YAML sections — but you lose the ability to name the variables.

Which means you end up with a form of automated variable naming like content1, content2, etc- which feels a little clunky.

[f0x2501]

@o-l-e right, so if you need a closing / ending why not implement a specific tag like

title: Lorem ipsum

content:

<md>

## This is my mind

Markdown content here ...
A colon appears in an awkward position within the text block   
Like: this.   

</md>

footer: this is the end

@kolber: whatever the solution, please let the standart yml identation syntax always work.

[o-l-e]

@mentat-fr and @kolber:
I kind of had the feeling that you needed to close a variable containing markdown once opening it.

From a users perspective, i love the ability to have as many markdown blocks as i want, and its variable named what i want. This is one of the things that is very appealing with Stacey. And with such a clear simple language.

I guess whatever final decision should fall on (for example):

• who is stacey for? (developer or "visual" types)
• if for the visual kind/user, what feels "easy" and in the spirit of stacey? (previous version)
• where is stacey going? (next level and more technical, or fewer steps to publishing)
• etc

Personally as the "visual" type, i have looked at others like Jekyll or similar things, but either they are too tricky to install(run shell scripts etc), or its too chopped up in bits and pieces(files here and there for any little line of text).

I know that from a programmers perspective reusing bits is what its all about, but the great thing about stacey is the simplicity and accessibility that it gives designers like myself, or other non-programming types. People i have suggested stacey to, absolutely love it. The dropbox like way of dropping stuff in folders, and the simple nature of markdown(that is growing as a standard very quickly) in plain .txt documents.

I think some of the new twig features are really nice, and see that stacey can be pushed so much further from a developers standpoint. But to be honest, i did not even know what a .yml document was until i read about it in this discussion group(not to say that it some day could become a standard).

I don't want to seem defensive or non progressive, but personally i just hope whatever solution, it is something "human", simple and fast.
That is what attracted me to stacey in the first place :)

[afonsoduarte]

I've been playing around with having .md (and .txt) files loading as variables and I quite like the result. It's a very easy solution to implement and separates quite neatly the page's properties (title, description, etc which go in the .yml file) from its content (images and texts). It's not a wasted separation, it's a useful and logical one.

Everyone who worked with it found it intuitive and easy to manage, and they definitely were on the visual type side of things.

[kariemil]

Has there been any kind of resolution on this issue?

[o-l-e]

@kariemil yes you can now use Markdown like so:

title: Page title

variable:
+++
Markdown here
+++