adding documentation (#48)

docs/develop/architecture.md

@@ -0,0 +1,34 @@
+# The MyST implementation architecture
+
+This page describes implementation details to help you understand the structure
+of the project.
+
+```{note}
+MyST currently relies on
+[a fork of the Mistletoe project](https://github.com/ExecutableBookProject/mistletoe).
+We hope to upstream these changes, but in the meantime make sure that you are using
+this fork.
+```
+
+## An extension to Mistletoe syntax
+
+At a high level, the MyST parser is an extension of the Mistletoe project. Mistletoe
+is a well-structured Python parser for CommonMark text. It also defines an extension
+point to include more syntax in parsed files. The MyST parser uses this extension
+point to define its own syntax options (e.g., for Sphinx roles and directives).
+
+The result of this parser is a Mistletoe AST.
+
+## A docutils renderer
+
+The MyST parser also defines a docutils renderer for the Mistletoe AST. This allows us
+to convert parsed elements of a MyST markdown file into docutils.
+
+## A Sphinx parser
+
+Finally, the MyST parser provides a parser for Sphinx, the documentation generation
+system. This parser does the following:
+
+* Parse markdown files with the MyST Mistletoe parser
+* Convert these files into docutils objects using the MyST docutils renderer
+* Provide these to Sphinx in order to use in building your site.

docs/develop/index.md

@@ -5,4 +5,5 @@ codebase, and some guidelines for how you can contribute.
 
 ```{toctree}
 contributing.md
+architecture.md
 ```

docs/examples/wealth_dynamics_md.md

@@ -1,5 +1,9 @@
 # Wealth Distribution Dynamics in MyST
 
+```{note}
+You can {download}`Download the source file for this page <./wealth_dynamics_md.md>`
+```
+
 In addition to what's in Anaconda, this lecture will need the following
 libraries:
 

docs/examples/wealth_dynamics_rst.rst

@@ -5,6 +5,12 @@
 Wealth Distribution Dynamics in rST
 ***********************************
 
+
+.. note::
+
+    You can
+    :download:`Download the source file for this page <./wealth_dynamics_rst.rst>`
+
 .. contents:: :depth: 2
 
 In addition to what's in Anaconda, this lecture will need the following libraries:

docs/index.md

@@ -1,31 +1,49 @@
-Welcome to myst's documentation!
-================================
+MyST - Markedly Structured Text
+===============================
 
-A little site to test out the Myst Sphinx parser. This will probably uncover
-some bugs, so please report them [here](https://github.com/ExecutableBookProject/meta/issues/24)
-if you find any more.
+MyST is a markdown flavor that implements the best parts of reStructuredText.
+It is a *slight* extension of CommonMark markdown.
 
-## Installing the Myst parser
+This project provides a parser for this flavor of markdown, as well as a bridge between
+MyST syntax and Sphinx. This allows for native markdown support for roles and
+directives.
 
-To install the myst parser (and thus to be able to build these docs),
-run the following:
-
-```bash
-pip install -e "git+https://github.com/ExecutableBookProject/myst_parser.git#egg=myst_parser[sphinx]"
+```{warn}
+The MyST parser is in an alpha stage, and may have breaking syntax to its implementation
+and to the syntax that it supports. Use at your own risk. If you find any issues,
+please report them
+[in the MyST issues](https://github.com/ExecutableBookProject/meta/issues/24)
 ```
 
-Or for package development:
-
-```bash
-git clone https://github.com/ExecutableBookProject/myst_parser
-cd myst_parser
-git checkout develop
-pip install -e .[sphinx,code_style,testing,rtd]
+## Why a new flavor of markdown?
+
+While markdown is ubiquitous, it is not powerful enough for writing modern,
+fully-featured documentation. Some flavors of markdown support features needed for this,
+but there is no community standard around various syntactic choices for these features.
+
+Sphinx is a documentation generation framework written in Python. It heavily-utilizes
+reStructuredText syntax, which is another markup language for writing documents. In
+particular, Sphinx defines two extension points that are extremely useful:
+in-line **roles** and block-level **directives**.
+
+**This project is an attempt at combining the simplicity and readability of Markdown
+with the power and flexibility of reStructuredText and the Sphinx platform.** It
+starts with the CommonMark markdown specification, and selectively adds a few extra
+syntax pieces to utilize the most powerful parts of reStructuredText.
+
+```{note}
+The CommonMark community has been discussing an "official" extension syntax for many
+years now (for example, see
+[this seven-year-old thread about directives](https://talk.commonmark.org/t/generic-directives-plugins-syntax/444) as well as
+[this more recent converstaion](https://talk.commonmark.org/t/support-for-extension-token/2771),
+and [this comment listing several more threads on this topic](https://talk.commonmark.org/t/extension-terminology-and-rules/1233)).
+
+We have chosen a "roles and directives" syntax that seems reasonable and follows other
+common conventions in Markdown flavors. However, if the CommonMark community ever
+decides on an "official" extension syntax, we will likely utilize this syntax for
+MyST.
 ```
 
-This should install the myst fork of mistletoe, along with the Sphinx parser
-that is included in the "extensions" configuration of this site.
-
 Here are the site contents:
 
 ```{toctree}

docs/using/index.md

@@ -4,5 +4,6 @@ The following pages are examples meant to highlight the functionality of
 Myst documents.
 
 ```{toctree}
+install.md
 syntax.md
 ```

docs/using/install.md

@@ -0,0 +1,24 @@
+# Installing the MyST Parser
+
+Installing the MyST parser provides access to two tools:
+
+* A MyST-to-docutils parser and renderer.
+* A Sphinx parser that utilizes the above tool in building your documenation.
+
+To install the MyST parser, run the following:
+
+```bash
+pip install -e "git+https://github.com/ExecutableBookProject/myst_parser.git#egg=myst_parser[sphinx]"
+```
+
+Or for package development:
+
+```bash
+git clone https://github.com/ExecutableBookProject/myst_parser
+cd myst_parser
+git checkout develop
+pip install -e .[sphinx,code_style,testing,rtd]
+```
+
+This should install the myst fork of mistletoe, along with the Sphinx parser
+that is included in the "extensions" configuration of this site.