Add syntax token tables

executablebooks · Mar 11, 2020 · b666b62 · b666b62
1 parent f733b66
commit b666b62
Show file tree

Hide file tree

Showing 3 changed files with 224 additions and 35 deletions.
diff --git a/docs/api/core_block_tokens.rst b/docs/api/core_block_tokens.rst
@@ -6,6 +6,80 @@ Core Block Tokens
 These block tokens are defined in the
 `CommonMark specification <https://spec.commonmark.org/0.29/>`_.
 
+.. list-table::
+    :header-rows: 1
+    :widths: 10 20 20
+
+    * - Token
+      - Description
+      - Example
+
+    * - ATX Heading
+      - Level 1-6 headings, denoted by number of ``#``
+      - .. code-block:: md
+
+           ### Heading level 3
+
+    * - SetextHeading
+      - Underlined header (using multiple ``=`` or ``-``)
+      - .. code-block:: md
+
+           Header
+           ======
+
+    * - Paragraph
+      - General inline text
+      - .. code-block:: md
+
+           any *text*
+
+    * - Quote
+      - quoted text
+      - .. code-block:: md
+
+           > this is a quote
+
+    * - BlockCode
+      - indented text (4 spaces or a tab)
+      - .. code-block:: md
+
+           included as literal *text*
+
+    * - CodeFence
+      - enclosed in 3 or more backticks with an optional language name
+      - .. code-block:: md
+
+           .. code-block:: python
+
+              print('this is python')
+
+    * - ThematicBreak
+      - Creates a horizontal line in the output
+      - .. code-block:: md
+
+           ---
+
+    * - List
+      - bullet points or enumerated.
+      - .. code-block:: md
+
+           - item
+           - nested item
+           1. numbered item
+
+    * - LinkDefinition
+      -  A substitution for an inline link, which can have a reference target (no spaces), and an optional title (in `"`)
+      - .. code-block:: md
+
+           [key]: https://www.google.com "a title"
+
+    * - HTMLBlock
+      - Any valid HTML (rendered in HTML output only)
+      - .. code-block:: html
+
+           <p>some text</p>
+
+
 Document
 ........
 

diff --git a/docs/api/core_span_tokens.rst b/docs/api/core_span_tokens.rst
@@ -6,6 +6,73 @@ Core Span Tokens
 These span tokens are defined in the
 `CommonMark specification <https://spec.commonmark.org/0.29/>`_.
 
+.. list-table::
+    :header-rows: 1
+    :widths: 10 20 20
+
+    * - Token
+      - Description
+      - Example
+    * - RawText
+      - any text
+      - .. code-block:: md
+
+           any text
+
+    * - EscapeSequence
+      - escaped symbols (to avoid them being interpreted as other syntax elements)
+      - .. code-block:: md
+
+           \*
+
+    * - LineBreak
+      - Soft or hard (ends with spaces or backslash)
+      - .. code-block:: md
+
+           A hard break\
+
+    * - Strong
+      - bold text
+      - .. code-block:: md
+
+           **strong**
+
+    * - Emphasis
+      - italic text
+      - .. code-block:: md
+
+           *emphasis*
+
+    * - InlineCode
+      - literal text
+      - .. code-block:: md
+
+           `a=1`
+
+    * - AutoLink
+      - link that is shown in final output
+      - .. code-block:: md
+
+           <http://www.google.com>
+
+    * - Link
+      - Reference ``LinkDefinitions``
+      - .. code-block:: md
+
+           [text](target "title") or [text][key]
+
+    * - Image
+      - link to an image
+      - .. code-block:: md
+
+           ![alt](src "title")
+
+    * - HTMLSpan
+      - any valid HTML (rendered in HTML output only)
+      - .. code-block:: html
+
+           <p>some text</p>
+
 Core
 ....
 
@@ -15,6 +82,28 @@ and extracts nested tokens.
 .. autoclass:: mistletoe.span_tokens.CoreTokens
     :show-inheritance:
 
+RawText
+.......
+
+.. autoclass:: mistletoe.span_tokens.RawText
+    :show-inheritance:
+
+EscapeSequence
+..............
+
+.. autoclass:: mistletoe.span_tokens.EscapeSequence
+    :members: pattern, parse_inner, parse_group
+    :undoc-members:
+    :show-inheritance:
+
+LineBreak
+.........
+
+.. autoclass:: mistletoe.span_tokens.LineBreak
+    :members: pattern, parse_inner, parse_group
+    :undoc-members:
+    :show-inheritance:
+
 Strong
 ......
 
@@ -36,12 +125,6 @@ InlineCode
     :show-inheritance:
 
 
-Image
-.....
-
-.. autoclass:: mistletoe.span_tokens.Image
-    :show-inheritance:
-
 Link
 ....
 
@@ -56,26 +139,10 @@ AutoLink
     :undoc-members:
     :show-inheritance:
 
-EscapeSequence
-..............
-
-.. autoclass:: mistletoe.span_tokens.EscapeSequence
-    :members: pattern, parse_inner, parse_group
-    :undoc-members:
-    :show-inheritance:
-
-LineBreak
-.........
-
-.. autoclass:: mistletoe.span_tokens.LineBreak
-    :members: pattern, parse_inner, parse_group
-    :undoc-members:
-    :show-inheritance:
-
-RawText
-.......
+Image
+.....
 
-.. autoclass:: mistletoe.span_tokens.RawText
+.. autoclass:: mistletoe.span_tokens.Image
     :show-inheritance:
 
 HTMLSpan

diff --git a/docs/api/extension_tokens.rst b/docs/api/extension_tokens.rst
@@ -8,6 +8,56 @@ These tokens do not form part of the core
 but are commonly implemented
 `extended syntax elements <https://www.markdownguide.org/extended-syntax/>`_.
 
+.. list-table::
+    :header-rows: 1
+    :widths: 10 20 20
+
+    * - Token
+      - Description
+      - Example
+    * - Strikethrough
+      - a line through the text
+      - .. code-block:: md
+
+           ~~some text~~
+
+    * - Math
+      - Dollar `$` enclosed math. Two ``$`` characters wrap multi-line math.
+      - .. code-block:: latex
+
+            $a=1$ $b=2$
+
+            $$
+            a=1
+            $$
+
+    * - FrontMatter
+      - A YAML block at the start of the document enclosed by ``---``
+      - .. code-block:: yaml
+
+           ---
+           key: value
+           ---
+
+    * - - Table
+        - TableRow
+        - TableCell
+      - Markdown table style, with pipe delimitation.
+      - .. code-block:: md
+
+           | a    | b    |
+           | :--- | ---: |
+           | c    | d    |
+
+    * - - Footnote
+        - FootReference
+      - A definition for a referencing footnote, that is placed at the bottom of the document.
+      - .. code-block:: md
+
+           Something that needs further explanation.[^ref]
+
+           [^ref]: Some footnote text
+
 Strikethrough
 .............
 
@@ -63,16 +113,6 @@ TableCell
     :show-inheritance:
     :exclude-members: __init__
 
-
-FootReference
-.............
-
-.. autoclass:: mistletoe.span_tokens_ext.FootReference
-    :members: pattern, parse_inner, parse_group
-    :undoc-members:
-    :show-inheritance:
-
-
 Footnote
 .........
 
@@ -81,3 +121,11 @@ Footnote
     :undoc-members:
     :show-inheritance:
     :exclude-members: __init__
+
+FootReference
+.............
+
+.. autoclass:: mistletoe.span_tokens_ext.FootReference
+    :members: pattern, parse_inner, parse_group
+    :undoc-members:
+    :show-inheritance: