Update docs with API (#3)

.circleci/config.yml

@@ -24,11 +24,11 @@ jobs:
           name: Build docs to store
           command: |
             cd docs
-            make html
+            make html-strict
 
       - store_artifacts:
           path: docs/_build/html/
-          destination: html-strict
+          destination: html
 
 
 workflows:

.readthedocs.yml

@@ -1,10 +1,10 @@
 version: 2
 
 python:
-  version: 3
-  install:
+   version: 3.7
+   install:
+      - requirements: docs/requirements.txt
       - method: pip
         path: .
         extra_requirements:
-          - rtd
-      - requirements: docs/requirements.txt
+            - rtd

README.md

@@ -28,10 +28,10 @@ deployed release that can be utilised by EBP. Here is a working list of 'up-stre
 - [x] Introduce `pre-commit` code linting and formatting: This standardizes the code style across the package, and ensures that new commits and Pull Requests also conform to it.
 - [x] Introduce `ReadTheDocs` documentation
 - [x] Add a conda-forge distribution of the package
-- [ ] Improve the AST API and documentation: I view [panflute](http://scorreia.com/software/panflute/index.html)'s implementation of the pandoc API in python, as the gold standard for how a pythonic AST API should be written and documented. Some tweaks to the current token class objects, and creating auto-generated RTD documentation, could achieve this.
+- [x] Improve the AST API and documentation: I view [panflute](http://scorreia.com/software/panflute/index.html)'s implementation of the pandoc API in python, as the gold standard for how a pythonic AST API should be written and documented. Some tweaks to the current token class objects, and creating auto-generated RTD documentation, could achieve this.
 - [ ] Storage of source line/column ranges: LSP and good rendering reporting of warnings/errors, requires access to the source line and column ranges of each parsed token.
 - [x] Asynchronous parsing: LSP requires documents to be parsed asynchronously. Currently, mistletoe contains a number of global state objects, which make parsing inherently not thread-safe. The simple solution to this is to store these items as `threading.local` objects. A related but slightly more complete solution is to introduce the idea of a 'scoped session', similar to that used by sqlalchemy for database access: [Contextual/Thread-local Sessions](https://docs.sqlalchemy.org/en/13/orm/contextual.html#unitofwork-contextual)
-- [ ] Improve extensibility of block tokens: A Markdown parser is inherently a Finite State-Machine + Memory (a.k.a Push-down Automata (PDA)), with parsing tokens as states (for a good example of a python state-machine see [pytransitions](https://github.com/pytransitions/transitions)). The problem with extensibility, is that inherently states are interdependent; when introducing a new state/token you must provide logic to all the other tokens, w.r.t to when to transition to this new token. Currently, MyST Parser sub-classes nearly all the Mistletoe block tokens to implement the extensions it requires, but it would be ideal if there was a more systematic approach for this.
+- [x] Improve extensibility of block tokens: A Markdown parser is inherently a Finite State-Machine + Memory (a.k.a Push-down Automata (PDA)), with parsing tokens as states (for a good example of a python state-machine see [pytransitions](https://github.com/pytransitions/transitions)). The problem with extensibility, is that inherently states are interdependent; when introducing a new state/token you must provide logic to all the other tokens, w.r.t to when to transition to this new token. Currently, MyST Parser sub-classes nearly all the Mistletoe block tokens to implement the extensions it requires, but it would be ideal if there was a more systematic approach for this.
 - [ ] Improve extensibility for span tokens: Mistletoe does allow for span token extensions to be added, at least in a simple way. However, as with block tokens above, there is often an interconnectivity to them, especially when considering nested span tokens. As of 7cc2c92, MyST-Parser now overrides some of Mistetoe's core logic to achieve correct parsing of Math tokens, but if possible this should be made more general.
 - [ ] Improve rendering logic: Currently, there is no concept of recursive walk-throughs or 'visitor' patterns in the Misteltoe `BaseRenderer`, which is a better method for rendering tree like structures (as used by docutils/panflute). Also, the current token instantiating (within context managers) needs improvement (see [miyuchina/mistletoe#56](https://github.com/miyuchina/mistletoe/issues/56)).
 

docs/api/base_elements.rst

@@ -0,0 +1,27 @@
+Base Elements
+-------------
+
+Token
+.....
+
+.. autoclass:: mistletoe.base_elements.Token
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+Block Token
+...........
+
+.. autoclass:: mistletoe.base_elements.BlockToken
+    :members:
+    :undoc-members:
+    :show-inheritance:
+
+
+Span Token
+..........
+
+.. autoclass:: mistletoe.base_elements.SpanToken
+    :members:
+    :undoc-members:
+    :show-inheritance:

docs/api/core_block_tokens.rst

@@ -0,0 +1,160 @@
+Core Block Tokens
+-----------------
+
+Document
+........
+
+.. autoclass:: mistletoe.block_tokens.Document
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+
+FrontMatter
+...........
+
+.. autoclass:: mistletoe.block_tokens.FrontMatter
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+
+Atx Heading
+...........
+
+.. autoclass:: mistletoe.block_tokens.Heading
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+
+Setext Heading
+..............
+
+.. autoclass:: mistletoe.block_tokens.SetextHeading
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+
+Paragraph
+.........
+
+.. autoclass:: mistletoe.block_tokens.Paragraph
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+
+Quote
+.....
+
+.. autoclass:: mistletoe.block_tokens.Quote
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+
+BlockCode
+.........
+
+.. autoclass:: mistletoe.block_tokens.BlockCode
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+
+CodeFence
+.........
+
+.. autoclass:: mistletoe.block_tokens.CodeFence
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+
+List
+....
+
+.. autoclass:: mistletoe.block_tokens.List
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+
+ListItem
+........
+
+.. autoclass:: mistletoe.block_tokens.ListItem
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+
+Table
+.....
+
+.. autoclass:: mistletoe.block_tokens.Table
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+
+TableRow
+........
+
+.. autoclass:: mistletoe.block_tokens.TableRow
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+
+TableCell
+.........
+
+.. autoclass:: mistletoe.block_tokens.TableCell
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+
+LinkDefinition
+..............
+
+.. autoclass:: mistletoe.block_tokens.LinkDefinition
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+
+ThematicBreak
+.............
+
+.. autoclass:: mistletoe.block_tokens.ThematicBreak
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+HTMLBlock
+.........
+
+.. autoclass:: mistletoe.block_tokens.HTMLBlock
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__

docs/api/core_span_tokens.rst

@@ -0,0 +1,101 @@
+Core Span Tokens
+----------------
+
+Strong
+......
+
+.. autoclass:: mistletoe.span_tokens.Strong
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+Emphasis
+.........
+
+.. autoclass:: mistletoe.span_tokens.Emphasis
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+InlineCode
+..........
+
+.. autoclass:: mistletoe.span_tokens.InlineCode
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+Strikethrough
+.............
+
+.. autoclass:: mistletoe.span_tokens.Strikethrough
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+Image
+.....
+
+.. autoclass:: mistletoe.span_tokens.Image
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+Link
+....
+
+.. autoclass:: mistletoe.span_tokens.Link
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+AutoLink
+.........
+
+.. autoclass:: mistletoe.span_tokens.AutoLink
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+EscapeSequence
+..............
+
+.. autoclass:: mistletoe.span_tokens.EscapeSequence
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+LineBreak
+.........
+
+.. autoclass:: mistletoe.span_tokens.LineBreak
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+RawText
+.......
+
+.. autoclass:: mistletoe.span_tokens.RawText
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__
+
+HTMLSpan
+.........
+
+.. autoclass:: mistletoe.span_tokens.HTMLSpan
+    :members:
+    :no-undoc-members:
+    :show-inheritance:
+    :exclude-members: __init__

docs/api/index.rst

@@ -0,0 +1,10 @@
+Mistletoe-EPB API
+=================
+
+.. toctree::
+    :maxdepth: 2
+    :caption: Core Tokens
+
+    base_elements.rst
+    core_block_tokens.rst
+    core_span_tokens.rst

docs/conf.py

@@ -67,7 +67,7 @@ def run_apidoc(app):
     logger.info("running apidoc")
     # get correct paths
     this_folder = os.path.abspath(os.path.dirname(os.path.realpath(__file__)))
-    api_folder = os.path.join(this_folder, "api")
+    api_folder = os.path.join(this_folder, "_api")
     module_path = os.path.normpath(os.path.join(this_folder, "../"))
     ignore_paths = ["../setup.py", "../test", "../contrib"]
     ignore_paths = [
@@ -99,7 +99,12 @@ def run_apidoc(app):
 }
 autodoc_member_order = "bysource"
 
-nitpick_ignore = []
+nitpick_ignore = [
+    ("py:class", "Tuple"),
+    ("py:class", "ForwardRef"),
+    ("py:class", "NoneType"),
+    ("py:class", "Any"),
+]
 
 
 def setup(app):

docs/index.md

@@ -72,6 +72,7 @@ maxdepth: 2
 caption: Contents
 ---
 using/index.md
+api/index.rst
 ```
 
 [ebp-link]: https://github.com/ExecutableBookProject

docs/requirements.txt

@@ -1 +1,4 @@
 git+https://github.com/pandas-dev/pandas-sphinx-theme.git@master
+attrs
+myst-parser
+pyyaml

mistletoe/__init__.py

@@ -2,7 +2,7 @@
 Make mistletoe easier to import.
 """
 
-__version__ = "0.9.2"
+__version__ = "0.9.3"
 __all__ = [
     "renderers",
     "base_elements",