Proposing a Macro standard for Markdown.

[howardroark]

Markdown should be readable

I have been thinking about the idea of a Macros for a long time. I feel it is inevitable that the use of Markdown and Git will increase in any field that involves professional writing. It solves so many of the peripheral problems that arise from the use of WSYIWIG editors and writing HTML directly. Git just makes sense because documents need to evolve over time. It's a perfect reason to ensure that your lines are less than 80 characters (or maybe an even better number). I see no reason to extend the core syntax to cater for any scenario outside of pure rational thought and argument. The core syntax should be readable and understandable in raw form by any reasonable person. As Albert Einstein said... “If you can't explain it to a six year old, you don't understand it yourself.”

There is going to be an increasing list of items that fall outside of this category. Many of which will need to be formatted in ways that go far beyond CSS and enter the realms of html canvas, svg, webgl... There should be a Macro standard that encourages people to extend Markdown in readable formats. If the job of Markdown is to output purely semantic html, all that really means is that it can be searchable by another human being... or readable.

If we follow the UNIX philosophy well we may want to argue that Macros should just be their own process that happens before or after the parsing of Markdown itself. I think that if the objective of both is to output HTML then it should happen at the same time to avoid security problems.

The DOM textContent of each Macro could be easily acted on after load to produce the extended visual result. Encourage the use of the ideas like GNU Bison when designing the syntax of each Macro extension. There is already a JavaScript variation called Jison.

This process would act as a way to protect the Markdown standard from becoming riddled with complex extensions. There is a lot of risk that Markdown starts being used as a tool for web development or expressly for programmers. If this starts to happen it could start to invalidate the potential for purely semantic HTML that Markdown has looked so promising to offer. A lot of consideration needs to be made towards the reality that we are trying to make information more searchable here.

An example

I would like to offer the idea as an example which is inspired by the project js-sequence-diagrams ...

Input

!!uml-sequence
Alice->Bob: Hello Bob, how are you?
Note right of Bob: Bob thinks
Bob-->Alice: I am good thanks!

# Always readable

Output

<div data-macro="uml-sequence" class="macro-uml-sequence">
Alice->Bob: Hello Bob, how are you?
Note right of Bob: Bob thinks
Bob-->Alice: I am good thanks!
</div>

<h1>Always readable</h1>

Some SVG work after the DOM load

Always readable

And writable... Markdown should focus on maintaining a natural flow of thought. The less often you have to wonder about how to do something, the less often you loose your concentration. The beauty of Markdown is that it just lets you keep typing, the syntax is so simple. Any extension should do its utmost to ensure a syntax that just makes sense to the user.

Another practical example

Suppose you wanted a Macro that could normalize any video link into an embedded player? Easy!

!!video
https://www.youtube.com/watch?v=JibxHpXqAfc

The DOM can make awesome things

If your goal is to turn Markdown into PDF files or anything related to printing you would be silly to not just build a solution that captures browser output with something like PhantomJS. You can build very smart solutions that use SVG and the block model. I had a little success in a quick attempt to port this idea to Markdown through HTTP GET parameters and image tags. It still has a bunch of bugs though and zero optimization. The point is that I'd rather just do some DOM work after the Markdown rendering. I want to build split screen editors that are smart and build off of ideas like GitHub's Jekyll.

Whitespace is informative

If it is a matter of compression, I just don't think there is real need. Mobile networks are getting faster and the cloud keep on getting cheaper. Why compress HTML? Why loose any subtle amount of semantic data that could be hidden in the whitespace? The main thing engineers need to worry about is the AMD JavaScript pattern that causes an excess of request cycles. You should always compress your JavaScript into one file, that is the whole purpose of it's lexical make up.

No need for inline

There is no reasonable reason to extend Markdown to support the idea of inline Macros. Outside of the agreed upon semantic features of writing itself nothing should need to exist inline with the text.

It's just a thought. Any consideration would be greatly appreciated!

[vyp]

Probably better off at http://talk.commonmark.org/.

• If we can access an AST directly, it would not be necessary to have ugly pre or post processors, for example for your js-sequence-diagrams example.
• PhantomJS is really nice but I suspect it is out of the scope of this specification. If commonmark becomes supported by pandoc, then pandoc could handle (direct or indirect) conversion to pdf easily enough anyway.
• HTML sanitization seems to be out of scope too.

[vyp]

Related to previous comment: #118 (comment)

[howardroark]

Probably better off at http://talk.commonmark.org/.

I really feel that keeping conversation in two place is a bad idea. I may be a troll, but I am going to be stubborn about this. I brought it up in #126.

If we can access an AST directly, it would not be necessary to have ugly pre or post processors, for example for your js-sequence-diagrams example.

Something like AST does sound excellent. I just don't see people who build UI really being able to grasp it or work with it easily. The Macro itself would most likely be a more visual representation of the language being expressed and should be accessible to developers who build cool UI.

PhantomJS is really nice but I suspect it is out of the scope of this specification. If commonmark becomes supported by pandoc, then pandoc could handle conversion to pdf easily enough anyway.

The point I made there is maybe a little off topic. Just an example of how you can leverage the DOM and UI developers to transform Markdown into whatever beautiful thing you need. Maybe you want to print a book and use HTML to generate each page. Pandoc could just ignore Macros all together or people could invent plugins for it that convert the Bison style syntax into visual elements.

HTML sanitization seems to be out of scope too.

Totally agree that HTML sanitization should be out of scope. Though you should at least be able to have a safe-mode option to turn off. I just updated the comment to include that.

[vyp]

I really feel that keeping conversation in two place is a bad idea. I may be a troll, but I am going to be stubborn about this. I brought it up in #126.

Yes sorry I did not see that prior to commenting. It's not up to me though, @jgm or one of the other core developers could easily just close this and insist on talk.commonmark regardless. The reasoning I think would be because some types of discussion/issues cannot be quickly solved and closed, that they require extended discussion to reach a solution, and thus should not clog up the 'issue' list at github, which is for more immediate issues.

Something like AST does sound excellent. I just don't see people who build UI really being able to grasp it or work with it easily. The Macro itself would most likely be a more visual representation of the language being expressed and should be accessible to developers who build cool UI.

Not sure if I agree with you here, since I feel an AST is a really easy and yet powerful and clean way to solve these things, and I'm not sure why those who build UI would be touching the content/backend side of it… but I was wondering if something like this relates to what you suggest? Although I have not followed that discussion too much so I am not completely sure if it is relevant here.

[howardroark]

I just want to make sure that this spec evolves to consider certain economic arguments. One being that the browser will always make sense to handle the bulk of the work. This is true from a server cost perspective and just from a contribution perspective. It should then be assumed that the most variety of code out there for the purpose of parsing Markdown will involve the DOM and JavaScript.

[jgm]

Hi! This kind of open-ended discussion is welcome, but it should go on
http://talk.commonmark.org, not on this tracker.

[howardroark]

Hey @jgm ... Sorry to be annoying here, but I need to take a bit of a stand re: #126

I did post to talk.commonmark.org, but it would not let me use an image because I was new to the system. You may go ahead and close this issue and I won't be offended in the least. I also apologize if any offence is taken by #126 ... I am only looking at this from the perspective of making Markdown better.

Thanks!

[howardroark]

I'm coming to grip with the reality that a Macro spec does not belong here. In reality it seems that only until John Gruber says so, can anything new really be added to Markdown. Not because there is some law, but more so because that is just how people think.

For now I'll keep plugging away here:
https://github.com/codingcoop/markdown-macros

Thanks!