Merge pull request #118 from bartvm/code_guidelines

Developer guidelines

docs/developer_guidelines.rst

@@ -6,9 +6,8 @@ ensure the codebase is of high quality, we ask all new developers to have a
 quick read through these rules to make sure that any code you contribute will be
 easy to merge!
 
-Code style
-----------
-
+Formatting guidelines
+---------------------
 Blocks follows the `PEP8 style guide`_ closely, so please make sure you are
 familiar with it. Our `Travis CI buildbot`_ runs flake8_ as part of every build,
 which checks for PEP8 compliance (using the pep8_ tool) and for some common
@@ -41,9 +40,92 @@ Blocks:
 .. _pyflakes: https://pypi.python.org/pypi/pyflakes
 .. _should be grouped: https://www.python.org/dev/peps/pep-0008/#imports
 
+Code guidelines
+---------------
+Some guidelines to keep in mind when coding for Blocks. Some of these are simply
+preferences, others stem from particular requirements we have e.g. in order to
+serialize training progress, support Python 2 and 3 simultaneously, etc.
+
+Validating function arguments
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+In general, be Pythonic and use `duck typing`_. That is, don't check the type of
+each argument given, because
+
+    When I see a bird that walks like a duck and swims like a duck and quacks
+    like a duck, I call that bird a duck.
+
+    -- James Whitcomb Riley
+
+If you need to perform some sort of input validation, don't use ``assert``
+statements. Raise a ``ValueError`` instead. ``assert`` statements should
+only be used for sanity tests i.e. they *should* never be triggered, unless
+there is a bug in the code.
+
+.. _duck typing: https://en.wikipedia.org/wiki/Duck_typing
+
+Abstract classes
+~~~~~~~~~~~~~~~~
+If a class is an `abstract base class`_, use Python's |abc|_ to mark it as such.
+
+.. code-block:: python
+
+   from abc import ABCMeta
+   from six import add_metaclass
+   @add_metaclass(ABCMeta)
+   class Abstract(object):
+       pass
+
+Our documentation generator (Sphinx_ with the autodoc_ extension, running on
+`Read the Docs`_) doesn't recognize classes which inherit the ``ABCMeta``
+metaclass as abstract and will try to instantiate them, causing errors when
+building documentation. To prevent this, make sure to always use the
+``add_metaclass`` decorator, regardless of the parent.
+
+.. _abstract base class: https://en.wikipedia.org/wiki/Class_%28computer_programming%29#Abstract_and_concrete
+.. |abc| replace:: ``abc``
+.. _abc: https://docs.python.org/3/library/abc.html
+.. _Sphinx: http://sphinx-doc.org/
+.. _autodoc: http://sphinx-doc.org/ext/autodoc.html
+.. _Read the Docs: https://readthedocs.org/
+
+Python 2 and 3
+~~~~~~~~~~~~~~
+Blocks aims to be both Python 2 and Python 3 compliant using a single code-base,
+without using 2to3_. There are many online resources which discuss the writing
+compatible code. For a quick overview see `the cheatsheet from Python
+Charmers`_. For non-trivial cases, we use the six_ compatibility library.
+
+Documentation should be written to be Python 3 compliant.
+
+.. _2to3: https://docs.python.org/2/library/2to3.html
+.. _the cheatsheet from Python Charmers: http://python-future.org/compatible_idioms.html
+.. _six: https://pythonhosted.org/six/
+
+Reraising exceptions
+~~~~~~~~~~~~~~~~~~~~
+When catching exceptions, use the :func:`~blocks.utils.reraise_as` function to
+reraise the exception (optionally with a new message or as a different type).
+Not doing so `clobbers the original traceback`_, making it impossible to use
+``pdb`` to debug the problems.
+
+.. _clobbers the original traceback: http://www.ianbicking.org/blog/2007/09/re-raising-exceptions.html
+
+Serialization
+~~~~~~~~~~~~~
+To ensure the reproducibility of scientific experiments Blocks tries to make
+sure that stopping and resuming training doesn't affect the final results. In
+order to do so it takes a radical approach, serializing the entire training
+state using dill_ (an extension of Python's native pickle_). Some things cannot
+be pickled, so their use should be avoided:
+
+* Generators
+* Dynamically generated classes (*possible but complicated*)
+
+.. _dill: http://trac.mystic.cacr.caltech.edu/project/pathos/wiki/dill
+.. _pickle: https://docs.python.org/3/library/pickle.html
+
 Docstrings
 ----------
-
 Blocks follows the `NumPy docstring standards`_. For a quick introduction, have
 a look at the NumPy_ or Napoleon_ examples of compliant docstrings. A few common
 mistakes to avoid:
@@ -60,7 +142,7 @@ rendered as you intended by looking at the `online documentation`_, which is
 automatically updated.
 
 Writing doctests_ is encouraged, and they are run as part of the test suite.
-They should be written to be Python 3 compliant.
+They should use Python 3 syntax.
 
 .. _NumPy docstring standards: https://github.com/numpy/numpy/blob/master/doc/HOWTO_DOCUMENT.rst.txt
 .. _NumPy: https://github.com/numpy/numpy/blob/master/doc/example.py
@@ -72,7 +154,6 @@ They should be written to be Python 3 compliant.
 
 Unit testing
 ------------
-
 Blocks uses unit testing to ensure that individual parts of the library behave
 as intended. It's also essential in ensuring that parts of the library are not
 broken by proposed changes.
@@ -84,6 +165,9 @@ coveralls_. Please make sure that at the very least your unit tests cover the
 core parts of your committed code. In the ideal case, all of your code should be
 unit tested.
 
+If you are fixing a bug, please be sure to add a unit test to make sure that the
+bug does not get re-intrduced later on.
+
 The test suite can be executed locally using nose2_ [#]_.
 
 .. [#] For all tests but the doctests, nose_ can also be used.
@@ -93,24 +177,3 @@ The test suite can be executed locally using nose2_ [#]_.
 .. _nose2: https://readthedocs.org/projects/nose2/
 .. _nose: http://nose.readthedocs.org/en/latest/
 
-Python 2 and 3
---------------
-
-Blocks aims to be both Python 2 and Python 3 compliant using a single code-base,
-without using 2to3_. There are many online resources which discuss the writing
-compatible code. For a quick overview see `the cheatsheet from Python
-Charmers`_. For non-trivial cases, we use the six_ compatibility library. Some
-quirks that this brings:
-
-* Our documentation generation framework, Sphinx_ (using autodoc_), will try to
-  initialize abstract classes if they inherit the ``ABCMeta`` class from a
-  parent. To make sure that the documentation builds successfully make sure to
-  use the ``@add_metaclass`` decorator on all abstract classes.
-
-Documentation should be written to be Python 3 compliant.
-
-.. _2to3: https://docs.python.org/2/library/2to3.html
-.. _the cheatsheet from Python Charmers: http://python-future.org/compatible_idioms.html
-.. _six: https://pythonhosted.org/six/
-.. _Sphinx: http://sphinx-doc.org/
-.. _autodoc: http://sphinx-doc.org/ext/autodoc.html

docs/index.rst

@@ -48,6 +48,8 @@ Getting started
 In-depth
 --------
 .. toctree::
+   :maxdepth: 2
+
    bricks_overview
    configuration
    developer_guidelines