docs: adding "Features" document

Comprehensive list of gluetool's features, complete with examples and
recordings.

.gitignore

@@ -16,6 +16,7 @@ dist
 **/.coverage*
 
 # auto-generated *.rst docs
+docs/source/modules.rst
 docs/source/gluetool.*
 
 # Ansible "retry" files

docs/source/conf.py

@@ -25,6 +25,7 @@
 import sys
 
 import sphinx.ext.autodoc
+import sphinx.ext.todo
 import sphinx.environment
 import sphinx_rtd_theme
 from docutils.utils import get_source_line
@@ -44,6 +45,7 @@
     'sphinx.ext.autodoc',
     'sphinx.ext.intersphinx',
     'sphinx.ext.coverage',
+    'sphinx.ext.todo',
     'sphinx.ext.viewcode'
 ]
 
@@ -85,7 +87,7 @@
 pygments_style = 'sphinx'
 
 # If true, `todo` and `todoList` produce output, else they produce nothing.
-todo_include_todos = False
+todo_include_todos = True
 
 
 # -- Options for HTML output ----------------------------------------------
@@ -107,6 +109,9 @@
 # so a file named "default.css" will overwrite the builtin "default.css".
 html_static_path = []
 
+html_logo = 'gluetool-logo-100.png'
+html_favicon = 'favicon.ico'
+
 
 # -- Options for HTMLHelp output ------------------------------------------
 

docs/source/features.rst

@@ -0,0 +1,35 @@
+``gluetool`` features
+=====================
+
+A comprehensive list of ``gluetool`` features, available helpers, tricks and tips. All the ways ``gluetool`` have to help module developers.
+
+.. include:: features/core.rst
+.. include:: features/help.rst
+.. include:: features/logging.rst
+.. include:: features/colors.rst
+.. include:: features/sentry.rst
+.. include:: features/utils.rst
+
+
+----
+
+Tool
+----
+
+.. todo::
+
+  Features yet to describe:
+
+  * reusable heart of gluetool
+  * config file on system level, user level or in a local dir
+
+
+----
+
+.. todo::
+
+  Features yet to describe:
+
+  * custom pylint checkers
+  * option names
+  * shared function definitions

docs/source/features/colors.rst

@@ -0,0 +1,96 @@
+Colorized output
+----------------
+
+``gluetool`` uses awesome `colorama <https://pypi.python.org/pypi/colorama>`_ library to enhance many of its outputs with colors. This is done in a transparent way, when developer does not need to think about it, and user can control this feature with a single option.
+
+
+Control
+~~~~~~~
+
+Color support is disabled by default, and can be turned on using ``--color`` option:
+
+.. raw:: html
+
+   <script src="https://asciinema.org/a/GhsulmjcL5FjXf5uA3XGE4sLi.js" id="asciicast-GhsulmjcL5FjXf5uA3XGE4sLi" async></script>
+
+
+If ``colorama`` package is **not** installed, color support cannot be turned on. If user tries to do that, ``gluetool`` will emit a warning:
+
+.. raw:: html
+
+   <script src="https://asciinema.org/a/NhorAwOzY3NiBhlT1QSXxpkqL.js" id="asciicast-NhorAwOzY3NiBhlT1QSXxpkqL" async></script>
+
+.. note::
+
+   As of now, ``colorama`` is ``gluetool``'s hard requirement, therefore it should not be possible - at least out of the box - to run ``gluetool`` wihout having ``colorama`` installed. However, this may change in the future, leaving this support up to user decision.
+
+To control this feature programatically, see :py:func:`gluetool.color.switch`.
+
+.. todo::
+
+   * ``seealso``:
+
+     * how to specify options
+
+Colorized logs
+~~~~~~~~~~~~~~
+
+Messages, logged on the terminal, are colorized based on their level:
+
+.. raw:: html
+
+   <script src="https://asciinema.org/a/170112.js" id="asciicast-170112" async></script>
+
+``DEBUG`` log level inherits default text color of your terminal, while, for example, ``ERROR`` is highlighted by being red, and ``INFO`` level is printed with nice, comforting green.
+
+.. todo::
+
+   * ``seealso``:
+
+     * logging
+
+Colorized help
+~~~~~~~~~~~~~~
+
+``gluetool`` uses reStructuredText (reST) to document modules, shared functions, opitons and other things, and to make the help texts even more readable, formatting, provided by reST, is enhanced with colors, to help users orient and focus on important information.
+
+.. raw:: html
+
+   <script src="https://asciinema.org/a/170118.js" id="asciicast-170118" async></script>
+
+.. todo::
+
+   * ``seealso``:
+
+     * generic help
+     * module help
+     * option help
+
+
+.. _colors-in-templates:
+
+Colors in templates
+~~~~~~~~~~~~~~~~~~~
+
+Color support is available for templates as well, via :py:func:`style <gluetool.color._style_colors>` filter.
+
+Example:
+
+.. code-block:: python
+   :emphasize-lines: 6
+
+   import gluetool
+
+   gluetool.log.Logging.create_logger()
+   gluetool.color.switch(True)
+
+   print gluetool.utils.render_template('{{ "foo" | style(fg="red", bg="green") }}')
+
+.. raw:: html
+
+   <script src="https://asciinema.org/a/170123.js" id="asciicast-170123" async></script>
+
+.. seealso::
+
+   :ref:`rendering-templates`
+       for more information about rendering templates with ``gluetool``.

docs/source/features/core.rst

@@ -0,0 +1,200 @@
+Core
+----
+
+
+.. _core-config-store:
+
+Module and ``gluetool`` configuration
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Configuration of ``gluetool`` and every module is gathered from different sources of different priorities, and merged into a single store, accessible by :py:meth:`option() <gluetool.glue.Configurable.option>` method. Configuration from later sources replaces values set by earlier sources, with lower priority. That way it is possible to combine multiple configuration files for a module, e.g. a generic site-wide configuration, with user-specific configuration overriding the global settings. Options specified on a command-line have the highest priority, overriding all configuration files.
+
+Consider following example module - it has just a single option, ``name``, whose value is logged in a form of greeting. The option has a default value, ``unknown being``:
+
+.. code-block:: python
+   :emphasize-lines: 7,11
+
+   from gluetool import Module
+
+   class M(Module):
+       name = 'dummy-module'
+
+        options = {
+            'name': {
+               'default': 'unknown being'
+            }
+        }
+
+        def execute(self):
+            self.info('Hi, {}!'.format(self.option('name')))
+
+.. raw:: html
+
+   <script src="https://asciinema.org/a/171492.js" id="asciicast-171492" async></script>
+
+With a configuration file, ``~/.gluetool.d/config/dummy-module``, you can change the value of ``name``:
+
+.. code-block:: ini
+
+   [default]
+   name = happz
+
+.. raw:: html
+
+   <script src="https://asciinema.org/a/171486.js" id="asciicast-171486" async></script>
+
+As you can see, configuration file for ``dummy-module`` is loaded and :py:meth:`option() <glue.Configurable.option>` method returns the correct value, ``happz``.
+
+Options specified on a command-line are merged into the store transparently, without any additional action necessary:
+
+.. raw:: html
+
+   <script src="https://asciinema.org/a/171487.js" id="asciicast-171487" async></script>
+
+.. todo::
+
+   * ``seealso``:
+
+     * options definitions
+
+.. seealso::
+
+   :ref:`core-config-files`
+       to see what configuration files are examined.
+
+.. _core-config-files:
+
+Configuration files
+~~~~~~~~~~~~~~~~~~~
+
+For every module - including ``gluetool`` itself as well - ``gluetool`` checks several possible sources of configuration, merging all information found into a single configuration store, which can be queried during runtime using :py:meth:`option() <gluetool.glue.Configurable.option>` method.
+
+Configuration files follow simple INI format, with a single section called ``[default]``, containing all options:
+
+.. code-block:: ini
+
+   [default]
+   option-foo = value bar
+
+.. warning::
+
+   Options can have short and long names (e.g. ``-v`` vs. ``--verbose``). Configuration files are using **only** the long option names to propagate their values to ``gluetool``. If you use a short name (e.g. ``v = yes``), such setting won't affect ``gluetool`` behavior!
+
+These files are checked for ``gluetool`` configuration:
+
+ * ``/etc/gluetool.d/gluetool``
+ * ``~/.gluetool.d/gluetool``
+ * ``./.gluetool.d/gluetool``
+ * options specified on a command-line
+
+These files are checked for module configuration:
+
+ * ``/etc/gluetool.d/config/<module name>``
+ * ``~/.gluetool.d/config/<module name>``
+ * ``./.gluetool.d/config/<module name>``
+ * options specified on a command-line
+
+If you're using a tool derived from ``gluetool``, it may add its own set of directories, e.g. using its name insead of ``gluetool``, but lists mentioned above should be honored by such tool anyway, to stay compatible with the base ``gluetool``.
+
+It is possible to change the list of directories, using ``--module-config-path`` option, the default list mentioned above is then replaced by directories provided by this option.
+
+.. todo::
+
+   * ``seealso``:
+
+     * option definitions
+
+.. seealso::
+
+   :ref:`core-config-store`
+       for more information on configuration handling.
+
+   :ref:`core-module-aliases`
+       for more information on module names and how to rename them
+
+
+.. _core-module-aliases:
+
+Module aliases
+~~~~~~~~~~~~~~
+
+Each module has a name, as set by its ``name`` class attribute, but sometimes it might be good to use the module under another name. Remember, the module configuration is loaded from files named just like the module, and if there's a way to "rename" module when used in different pipelines, user might use different configuration files for the same module.
+
+Consider following example module - it has just a single option, ``name``, whose value is logged in a form of greeting:
+
+.. code-block:: python
+   :emphasize-lines: 4
+
+   from gluetool import Module
+
+   class M(Module):
+       name = 'dummy-module'
+
+        options = {
+            'name': {}
+        }
+
+        def execute(self):
+            self.info('Hi, {}!'.format(self.option('name')))
+
+With the following configuration, ``~/.gluetool.d/config/dummy-module``, it will greet your users in a more friendly fashion:
+
+.. code-block:: ini
+
+   [default]
+   name = handsome gluetool user
+
+.. raw:: html
+
+   <script src="https://asciinema.org/a/172536.js" id="asciicast-172536" async></script>
+
+For some reason, you might wish to use the module in another pipeline, sharing the configuration between both pipelines, but you want to change the greeted entity. One option is to use a command-line option, which overrides configuration files but that would make one of your pipelines a bit exceptional, having some extra command-line stuff. Other way is to tell ``gluetool`` to use the module but give it a different name. Add the extra configuration file for your "renamed" module, ``~/.gluetool.d/config/customized-dummy-module``:
+
+.. code-block:: ini
+
+   [default]
+   name = beautiful
+
+.. raw:: html
+
+   <script src="https://asciinema.org/a/172537.js" id="asciicast-172537" async></script>
+
+Module named ``customized-dummy-module:dummy-module`` does not exist but this form tells ``gluetool`` it should create an instance of ``dummy-module`` module, and name it ``customized-dummy-module``. This is **the** name used to find and load module's configuration.
+
+You may combine aliases and original modules as much as you wish - ``gluetool`` will keep track of names and the actual modules, and it will load the correct configuration:
+
+.. raw:: html
+
+   <script src="https://asciinema.org/a/172540.js" id="asciicast-172540" async></script>
+
+
+----
+
+.. todo::
+
+  Features yet to describe:
+
+  * system-level, user-level and local dir configs
+  * configurable list of module paths (with default based on sys.prefix)
+  * dry-run support
+  * controled by core
+  * module can check what level is set, and take corresponding action. core takes care of logging
+  * exception hierarchy
+  * hard vs soft errors
+  * chaining supported
+  * custom sentry fingerprint and tags
+  * Failure class to pass by internally
+  * processes config file, command line options
+  * argparser to configure option
+  * short/long options
+  * option groups
+  * required options
+  * note to print as a part of help
+  * eval context
+  * shared functions
+  * overloaded shared
+  * require_shared
+  * module logging helpers
+  * sanity => execute => destroy - pipeline flow
+  * failure access
+  * module discovery mechanism