Markdown or rST?

[matin]

Should the API specs for Balanced be written in GitHub Flavored Markdown or reStructuredText?

In creating this repo, we were having an internal debate around whether we should use Markdown or rst. Balanced is a Python shop, and it's considerably easier for us to generate rst. That means it's also easier to ensure validation and consistancy. However, the specs are being written for GitHub, and Markdown is the standard on GitHub.

[bninja]

rst seems better suited for spec docs which will likely feature things, such as tables, that md intentionally doesn't support (or requires that you drop down to html to achieve).

For reading that probably doesn't matter as the tedious work of generating say a table in html should be entirely automated when building the spec from the implementation. Additionally markdown is clearly favored by github (so e.g. nicer rendering, shortcuts, comments and issues are all .md).

For writing it does matter. The idea is for people to edit the markup files directly to show how things should be done and being forced to drop down to html to do basic things is annoying. There are probably tools for markdown that make this easier.

If you identify all the structural features you need to support and find that markdown supports them as first class markup elements then it would probably fine.

This thread raises may good points of which is preferable, when and why:

http://stackoverflow.com/questions/34276/markdown-versus-restructuredtext

In this case I think rst should win.

[matin]

@whit537 this repo largely came out of a conversation you and I had. Thoughts?

[matin]

ping @alexnotov, @bcm, @khussein, and @ajsharp

[zealoushacker]

I am going to basically point to github's own API repo as an example of stuff that can be done with gfm: http://developer.github.com/v3/. After a cursory look, I think they use a static layout file and then pull in .md content. (I may be mistaken).

I think it's a matter of goals...
If your goal is to generate the docs for the larger community to help you collaborate and edit the docs, then I would suggest gfm (because it is the preference of the community).
If your goal is to have a private repo, where you maintain your API docs in version control and don't have the larger github community participate, then use whatever you prefer (rst, or otherwise).

my humble, but very opinionated 2cents =)

[bcm]

for what it's worth, I'm very used to writing gfm for technical prose and api docs, and I have never missed the lack of table syntax. I've had to drop into html maybe a half dozen times, and it was never a pain. also, I sure do like how that github site looks.

[zealoushacker]

@bcm 👍 regarding "never missed the lack of table syntax".

I've dug a bit deeper into how github itself maintains its API docs. So 95% of their docs are written in gfm. They use nanoc to compile the site into static html from their .md files and whatever other formats they use.

@matin You and your devs have probably read this thread and may even know better alternatives...

[matin]

@alexnotov I'm a supporter of Markdown, especially GFM. I think @mahmoudimus, @bninja, and @msherry need to elaborate on their case for rST.

[mahmoudimus]

I realized that most of my arguments here were a little incohesive, so I've deleted them and added them into one post. First and foremost, I want to stress that there are serveral use cases for the power of reST when designing something like an API spec that has to be self validating. What does it mean to be self-validating? It means that the documentation is auto-generated and checked against the current version of the "master" branch of the API spec (in this case, it's the balanced-api repository).

Here are some technical reasons why I think github flavored markdown is a poor choice for auto-generated api specification from python source code:

1. Not having the list directive, which controls tables and their respective layouts, restricts the way the documentation generates its fields (i.e. look @ App.Net's spec - it is hand curated w/ custom HTML and no ability to auto-validate their API: https://github.com/appdotnet/api-spec).
2. Markdown is notorioius for having limitations, like code in lists, or finicky stuff that require you to dive into HTML (like images inside links).
3. GitHub's support for reST is just as great as GFM, but has the ability to be better.
4. Sphinx's ecosystem and extensive docutils plugins.
5. Markdown's intention was for simple marked up files like READMEs or simple documents -- not an API spec.

Given reStructuredText's extensibility, it is clearly superior to add different domains / directives that allow customized rendering of the way the API spec is set for design. It also allows us to keep our API spec in sync w/ the API spec the master branch dictates

[chadwhitacre]

@matin Love the repo, very excited to see this opening up. :-)

The primary concern is excellent end-user experience, not tools. In this case, the end-user is the developer who is using the documentation. I'm with @bcm: "I sure do like how that github site looks." Achieving that level of excellence is a function of taste and competence and hard work, which depend primarily on the people doing the work, and not necessarily on the tools employed. However, tool choice can be a barometer of taste and competence, as Linus famously, um, observed. ;-)

My personal strategy when I don't feel competent (as here) is to find examples of excellence and attempt to emulate them. In this case I see plenty of people writing reST or even creole, but the user experiences over there are rather more mundane than on GitHub, I daresay. The fact that I think so highly of GitHub in general predisposes me towards GFM, in the absence of a well-formed opinion otherwise regarding structured text formats.

What does it mean to be self-validating? It means that the documentation is auto-generated and checked against the current version of the "master" branch of the API spec (in this case, it's the balanced-api repository).

@mahmoudimus Can you help me understand this better? The idea is that the documentation is maintained in a literate style inside docstrings? What does the checking step entail? Are you validating behavior of the API against documented behavior? Or are you validating auto-generated documentation against manually maintained documentation in the balanced-api repo? What does the balanced-api repo contain? Documentation?

[bcm]

this: "The fact that I think so highly of GitHub in general predisposes me
towards GFM, in the absence of a well-formed opinion otherwise regarding
structured text formats."

I have zero experience with reST so can't comment on how powerful or easy
to use it might be. my main concerns when choosing an authoring tool are
how easy it is for me to write content and how attractive and easily
navigable the output is. I don't typically consider automated publishing
features because I don't typically use them. I've always maintained these
kinds of documents by hand simply because I prefer to have 100% control
over the content. therefore, my experience and opinion might not be super
useful in this situation :)

[matin]

Here are the two arguments I get out of this conversation:

• GFM is the GitHub standard. It adopts part of rST to make it more powerful, but it's the standard nonetheless, which makes it easier for anyone to make changes and issue pull requests
• rST is more powerful and easier to generate from Python because of docutils. Automated validation keeps Balanced honest in guaranteeing that the API is consistent with this repo. Even if GFM was generated, it would happen with rST as an intermediate anyways

I propose we move forward with rST for all generated content. It allows Balanced to move forward more quickly, and we can revisit later.

GFM will be used for all handwritten content like the TOCs in READMEs and proposals inside the repo that are not in GitHub issues obviously.

I'm not a huge fan of compromise, or splitting across two different technologies. However, this allows us to move forward.

Final thoughts before I close the issue?

[mahmoudimus]

@whit537 @bcm Thanks for your comments! I completely agree with you both, Github's documentation looks awesome. However, I have to ask @matin to elaborate on how he wants to display the API spec before I can discuss this further.

• Does @matin want the api spec to be viewable from github's repository viewer?
• Does @matin want to host the api spec on a github page?

This will help make the goal of this discussion clearer as those two choices will open up other paths of thought.

@whit537, as a reference for great, navigatable, aesthetically pleasing documentation written in reST, take a look at Flask's documentation.

@whit537, we leverage Sphinx's python parsing tools, extend it to understand our in house tools, and use the auto-doc module to spit out a very maintainable, reflective specification of the current api code. We do not do much manual orientiation for a few reasons:

• avoid cognitive overhead of developers adding a field, restriction, or changing any of the guarnatees we're trying to make with Balanced.
• if a system is guaranteed to reproduce its specifications programmatically, it follows that an external automated system can also validate these specifications by comparing them to a set of externally available automatically generated master specifications -- providing guarantees on interfaces.

[mahmoudimus]

@matin just saw your comments. If you have a moment to answer the two questions I posted above, I'd love to make another point.

[matin]

Does @matin want the api spec to be viewable from github's repository viewer?

Yes. Definitely.

Does @matin want to host the api spec on a github page?

No. We can publish the docs as a separate repo like GitHub has, but using GitHub for hosting, I believe, limits us to static content, which means no server side search.

[mahmoudimus]

@matin thanks for your comments and clarifications.

Since that is what your intention is, it is important to note that the aesthetic pleasing features of Github's API documentation can not be taken into comparison in this discussion, but can be used as a point of reference, since their documentation is generated for a Github page, see https://github.com/github/developer.github.com.

Since there's a pre-compilation process to generate this page, Github leverages CSS and many other web-based frameworks to generate their documentation. It is indeed NOT intended for viewing it from a Github repository, see https://github.com/github/developer.github.com/blob/master/content/v3/issues.md#response as an example.

Notice the explicit:

Response

<%= headers 200, :pagination => true %> <%= json(:issue) { |h| [h] } %>

If the intention is to leverage Github's repository viewer stands for the balanced-api specification repository, then I believe I've demonstrated enough evidence that GFM does not provide the correct ecosystem to support what we're trying to accomplish.

I'd like to also note that Github's TODO in their README states:

TODO
    - Integrate through a simple hurl.it app for live API calls.
    - Maybe add a nice TOC at the top of each page.
    - Write a task for verifying JSON Resource examples against the actual API.

The third TODO is particularly what we're trying to solve for.

[matin]

@mahmoudimus just to understand clearly. Are you arguing against using a mix of Markdown and rST including for handwritten content in READMEs?

Here's an example of someone outside of the Balanced team doing a pull request to manually written content: #4

[bcm]

just to throw one more issue onto the table - whatever system you choose,
please endeavor to make it capable of generating comprehensive
documentation for all api calls, clearly indicating optional and required
parameters, their allowed values, etc. the current documentation is
significantly lacking in that regard, although I can't tell from my vantage
point if that's due to limitations of your current process or simply
because you haven't gotten around to documenting some parts of the system.
thanks!

On Fri, Aug 17, 2012 at 10:56 AM, Matin Tamizi notifications@github.comwrote:

@mahmoudimus https://github.com/mahmoudimus just to understand clearly.
Are you arguing against using a mix of Markdown and rST including for
handwritten content in READMEs?

Here's an example of someone outside of the Balanced team doing a pull
request to manually written content:
#4

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/1#issuecomment-7830126.

[matin]

@bninja wrote an initial version with rST. Can you issue a pull request and reference this issue to provide an example?

[chadwhitacre]

How does this balanced-api repo relate to, e.g., https://www.balancedpayments.com/docs/python? Are they the same or different?

[matin]

@whit537 it would be used as the content for https://www.balancedpayments.com/docs/api

It would be great to publish the entire docs site on GitHub, especially for tracking the changelog, but we'll have to revisit that later. I really like how GitHub did this for their own docs.

[bninja]

@matin submitted pull request showing what it would roughly look like using rst

[matin]

For now, we'll use rST for all generated content like this, and all handwritten content will be written in GFM. This let's us move forward more quickly, and we can revisit later if necessary.

[mjallday]

http://www.sphinx-doc.org/en/stable/markdown.html

IT.
HAS.
BEEN.
SOLVED>

Sphinx FTW