fix docs and add docstring check

.github/workflows/lint_and_docs.yml

@@ -16,15 +16,14 @@ jobs:
         python -m pip install --upgrade pip setuptools wheel
     - name: Install dependencies
       run: |
-        python -m pip install flake8
+        python -m pip install ".[dev]" --upgrade
     - name: Lint with flake8
       run: |
         flake8 . --count --show-source --statistics
-    - name: Install dependencies
-      run: |
-        pip install ".[dev]" --upgrade
     - name: Documentation test
       run: |
+        pydocstyle tianshou
+        doc8 docs --max-line-length 1000
         cd docs
         make html SPHINXOPTS="-W"
         cd ..

.github/workflows/profile.yml

@@ -16,7 +16,7 @@ jobs:
         python -m pip install --upgrade pip setuptools wheel
     - name: Install dependencies
       run: |
-        pip install ".[dev]" --upgrade
+        python -m pip install ".[dev]" --upgrade
     - name: Test with pytest
       run: |
         pytest test/throughput --durations=0 -v

.github/workflows/pytest.yml

@@ -26,7 +26,7 @@ jobs:
         python -m pip install --upgrade pip setuptools wheel
     - name: Install dependencies
       run: |
-        pip install ".[dev]" --upgrade
+        python -m pip install ".[dev]" --upgrade
     - name: Test with pytest
       # ignore test/throughput which only profiles the code
       run: |

README.md

@@ -40,6 +40,7 @@ Here is Tianshou's other features:
 - Support customized training process [Usage](https://tianshou.readthedocs.io/en/latest/tutorials/cheatsheet.html#customize-training-process)
 - Support n-step returns estimation and prioritized experience replay for all Q-learning based algorithms; GAE, nstep and PER are very fast thanks to numba jit function and vectorized numpy operation
 - Support multi-agent RL [Usage](https://tianshou.readthedocs.io/en/latest/tutorials/cheatsheet.html##multi-agent-reinforcement-learning)
+- Comprehensive documentation, PEP8 code-style checking, type checking and [unit tests](https://github.com/thu-ml/tianshou/actions)
 
 In Chinese, Tianshou means divinely ordained and is derived to the gift of being born with. Tianshou is a reinforcement learning platform, and the RL algorithm does not learn from humans. So taking "Tianshou" means that there is no teacher to study with, but rather to learn by themselves through constant interaction with the environment.
 

docs/conf.py

@@ -23,9 +23,9 @@
 
 # -- Project information -----------------------------------------------------
 
-project = 'Tianshou'
-copyright = '2020, Tianshou contributors.'
-author = 'Tianshou contributors'
+project = "Tianshou"
+copyright = "2020, Tianshou contributors."
+author = "Tianshou contributors"
 
 # The full version, including alpha/beta/rc tags
 release = version
@@ -37,51 +37,61 @@
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
 extensions = [
-    'sphinx.ext.autodoc',
-    'sphinx.ext.doctest',
-    'sphinx.ext.intersphinx',
-    'sphinx.ext.coverage',
+    "sphinx.ext.autodoc",
+    "sphinx.ext.doctest",
+    "sphinx.ext.intersphinx",
+    "sphinx.ext.coverage",
     # 'sphinx.ext.imgmath',
-    'sphinx.ext.mathjax',
-    'sphinx.ext.ifconfig',
-    'sphinx.ext.viewcode',
-    'sphinx.ext.githubpages',
-    'sphinxcontrib.bibtex',
+    "sphinx.ext.mathjax",
+    "sphinx.ext.ifconfig",
+    "sphinx.ext.viewcode",
+    "sphinx.ext.githubpages",
+    "sphinxcontrib.bibtex",
 ]
 
 # Add any paths that contain templates here, relative to this directory.
-templates_path = ['_templates']
-source_suffix = ['.rst', '.md']
-master_doc = 'index'
+templates_path = ["_templates"]
+source_suffix = [".rst", ".md"]
+master_doc = "index"
 
 # List of patterns, relative to source directory, that match files and
 # directories to ignore when looking for source files.
 # This pattern also affects html_static_path and html_extra_path.
-exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
-autodoc_default_options = {'special-members': ', '.join([
-    '__len__', '__call__', '__getitem__', '__setitem__',
-    '__getattr__', '__setattr__'])}
+exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
+autodoc_default_options = {
+    "special-members": ", ".join(
+        [
+            "__len__",
+            "__call__",
+            "__getitem__",
+            "__setitem__",
+            # "__getattr__",
+            # "__setattr__",
+        ]
+    )
+}
 
 # -- Options for HTML output -------------------------------------------------
 
 # The theme to use for HTML and HTML Help pages.  See the documentation for
 # a list of builtin themes.
 #
-html_theme = 'sphinx_rtd_theme'
+html_theme = "sphinx_rtd_theme"
 html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
 
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
-html_static_path = ['_static']
+html_static_path = ["_static"]
 
-html_logo = '_static/images/tianshou-logo.png'
+html_logo = "_static/images/tianshou-logo.png"
 
 
 def setup(app):
     app.add_js_file("js/copybutton.js")
     app.add_css_file("css/style.css")
 
+
 # -- Extension configuration -------------------------------------------------
 
 # -- Options for intersphinx extension ---------------------------------------

docs/contributing.rst

@@ -1,6 +1,7 @@
 Contributing to Tianshou
 ========================
 
+
 Install Develop Version
 -----------------------
 
@@ -16,6 +17,7 @@ in the main directory. This installation is removable by
 
     $ python setup.py develop --uninstall
 
+
 PEP8 Code Style Check
 ---------------------
 
@@ -25,6 +27,7 @@ We follow PEP8 python code style. To check, in the main directory, run:
 
     $ flake8 . --count --show-source --statistics
 
+
 Test Locally
 ------------
 
@@ -34,6 +37,7 @@ This command will run automatic tests in the main directory
 
     $ pytest test --cov tianshou -s --durations 0 -v
 
+
 Test by GitHub Actions
 ----------------------
 
@@ -54,6 +58,7 @@ Test by GitHub Actions
 .. image:: _static/images/action3.png
     :align: center
 
+
 Documentation
 -------------
 
@@ -70,3 +75,28 @@ To compile documentation into webpages, run
 under the ``docs/`` directory. The generated webpages are in ``docs/_build`` and can be viewed with browsers.
 
 Chinese documentation is in https://tianshou.readthedocs.io/zh/latest/.
+
+
+Documentation Generation Test
+-----------------------------
+
+We have the following three documentation tests:
+
+1. pydocstyle: test docstrings under ``tianshou/``. To check, in the main directory, run:
+
+.. code-block:: bash
+
+    $ pydocstyle tianshou
+
+2. doc8: test ReStructuredText formats. To check, in the main directory, run:
+
+.. code-block:: bash
+
+    $ doc8 docs
+
+3. sphinx test: test if there is any errors/warnings when generating front-end html documentations. To check, in the main directory, run:
+
+.. code-block:: bash
+
+    $ cd docs
+    $ make html SPHINXOPTS="-W"

docs/index.rst

@@ -3,6 +3,7 @@
    You can adapt this file completely to your liking, but it should at least
    contain the root `toctree` directive.
 
+
 Welcome to Tianshou!
 ====================
 
@@ -25,14 +26,15 @@ Here is Tianshou's other features:
 
 * Elegant framework, using only ~2000 lines of code
 * Support parallel environment simulation (synchronous or asynchronous) for all algorithms: :ref:`parallel_sampling`
-* Support recurrent state/action representation in actor network and critic network (RNN-style training for POMDP): :ref:`rnn_training`
-* Support any type of environment state (e.g. a dict, a self-defined class, ...): :ref:`self_defined_env`
-* Support customized training process: :ref:`customize_training`
+* Support recurrent state representation in actor network and critic network (RNN-style training for POMDP): :ref:`rnn_training`
+* Support any type of environment state/action (e.g. a dict, a self-defined class, ...): :ref:`self_defined_env`
+* Support :ref:`customize_training`
 * Support n-step returns estimation :meth:`~tianshou.policy.BasePolicy.compute_nstep_return` and prioritized experience replay :class:`~tianshou.data.PrioritizedReplayBuffer` for all Q-learning based algorithms; GAE, nstep and PER are very fast thanks to numba jit function and vectorized numpy operation
-* Support multi-agent RL: :doc:`/tutorials/tictactoe`
+* Support :doc:`/tutorials/tictactoe`
 
 中文文档位于 `https://tianshou.readthedocs.io/zh/latest/ <https://tianshou.readthedocs.io/zh/latest/>`_
 
+
 Installation
 ------------
 
@@ -70,6 +72,7 @@ If no error occurs, you have successfully installed Tianshou.
 
 Tianshou is still under development, you can also check out the documents in stable version through `tianshou.readthedocs.io/en/stable/ <https://tianshou.readthedocs.io/en/stable/>`_.
 
+
 .. toctree::
    :maxdepth: 1
    :caption: Tutorials
@@ -81,6 +84,7 @@ Tianshou is still under development, you can also check out the documents in sta
    tutorials/trick
    tutorials/cheatsheet
 
+
 .. toctree::
    :maxdepth: 1
    :caption: API Docs
@@ -92,6 +96,7 @@ Tianshou is still under development, you can also check out the documents in sta
    api/tianshou.exploration
    api/tianshou.utils
 
+
 .. toctree::
    :maxdepth: 1
    :caption: Community

docs/tutorials/batch.rst

@@ -3,9 +3,10 @@
 Understand Batch
 ================
 
-:class:`~tianshou.data.Batch` is the internal data structure extensively used in Tianshou. It is designed to store and manipulate hierarchical named tensors. This tutorial aims to help users correctly understand the concept and the behavior of ``Batch`` so that users can make the best of Tianshou.
+:class:`~tianshou.data.Batch` is the internal data structure extensively used in Tianshou. It is designed to store and manipulate hierarchical named tensors. This tutorial aims to help users correctly understand the concept and the behavior of :class:`~tianshou.data.Batch` so that users can make the best of Tianshou.
+
+The tutorial has three parts. We first explain the concept of hierarchical named tensors, and introduce basic usage of :class:`~tianshou.data.Batch`, followed by advanced topics of :class:`~tianshou.data.Batch`.
 
-The tutorial has three parts. We first explain the concept of hierarchical named tensors, and introduce basic usage of ``Batch``, followed by advanced topics of ``Batch``.
 
 Hierarchical Named Tensors
 ---------------------------
@@ -43,11 +44,13 @@ Note that, storing hierarchical named tensors is as easy as creating nested dict
 
 The real problem is how to **manipulate them**, such as adding new transition tuples into replay buffer and dealing with their heterogeneity. ``Batch`` is designed to easily create, store, and manipulate these hierarchical named tensors.
 
+
 Basic Usages
 ------------
 
 Here we cover some basic usages of ``Batch``, describing what ``Batch`` contains, how to construct ``Batch`` objects and how to manipulate them.
 
+
 What Does Batch Contain
 ^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -69,6 +72,7 @@ The content of ``Batch`` objects can be defined by the following rules.
 
     The data types of tensors are bool and numbers (any size of int and float as long as they are supported by NumPy or PyTorch). Besides, NumPy supports ndarray of objects and we take advantage of this feature to store non-number objects in ``Batch``. If one wants to store data that are neither boolean nor numbers (such as strings and sets), they can store the data in ``np.ndarray`` with the ``np.object`` data type. This way, ``Batch`` can store any type of python objects.
 
+
 Construction of Batch
 ^^^^^^^^^^^^^^^^^^^^^
 
@@ -136,6 +140,7 @@ There are two ways to construct a ``Batch`` object: from a ``dict``, or using ``
 
    </details><br>
 
+
 Data Manipulation With Batch
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -285,11 +290,13 @@ Stacking and concatenating multiple ``Batch`` instances, or split an instance in
 
    </details><br>
 
+
 Advanced Topics
 ---------------
 
 From here on, this tutorial focuses on advanced topics of ``Batch``, including key reservation, length/shape, and aggregation of heterogeneous batches.
 
+
 .. _key_reservations:
 
 Key Reservations
@@ -347,6 +354,7 @@ The ``Batch.is_empty`` function has an option to decide whether to identify dire
 
     Do not get confused with ``Batch.is_empty`` and ``Batch.empty``. ``Batch.empty`` and its in-place variant ``Batch.empty_`` are used to set some values to zeros or None. Check the API documentation for further details.
 
+
 Length and Shape
 ^^^^^^^^^^^^^^^^
 
@@ -391,6 +399,7 @@ The ``obj.shape`` attribute of ``Batch`` behaves somewhat similar to ``len(obj)`
 
 4. The shape of reserved keys is undetermined, too. We treat their shape as ``[]``.
 
+
 .. _aggregation:
 
 Aggregation of Heterogeneous Batches
@@ -457,6 +466,7 @@ For a set of ``Batch`` objects denoted as :math:`S`, they can be aggregated if t
 
 The ``Batch`` object ``b`` satisfying these rules with the minimum number of keys determines the structure of aggregating :math:`S`. The values are relatively easy to define: for any key chain ``k`` that applies to ``b``, ``b[k]`` is the stack/concatenation of ``[bi[k] for bi in S]`` (if ``k`` does not apply to ``bi``, the appropriate size of zeros or ``None`` are filled automatically). If ``bi[k]`` are all ``Batch()``, then the aggregation result is also an empty ``Batch()``.
 
+
 Miscellaneous Notes
 ^^^^^^^^^^^^^^^^^^^