lots of documentation updates and refactoring

doc/about.rst

@@ -13,19 +13,31 @@ used - and the overall workflow management. Both are automatically determined
 from the succint, declarative-like interfaces of the model processes.
 
 Next versions of xarray-simlab will hopefully handle other technical issues
-like logging simulation progress, command line integration and running (many)
-simulations in parallel.
+like logging simulation progress, a plug-in system for custom co-processors
+(e.g., for interactive visualization), command line integration and/or running
+many simulations in parallel (for example in the context of sensitivity analyses
+or inversion procedures).
 
 Motivation
 ----------
 
 xarray-simlab is being developped with the idea of reducing the gap between the
 environments used for building and running computational models and the ones
-used for processing and analysing simulation results. It also encourages
-building new models from re-usable components and avoid reinventing the wheel.
+used for processing and analyzing simulation results. If the latter environments
+become more powerful and interactive, progress has still to be done for the
+former ones.
 
-The design of this tool is mainly focused on fast model development and easy
-creation of various simulation settings. Ultimately, this would optimize the
+xarray-simlab also encourages building new models from re-usable sets of
+components in order to avoid reinventing the wheel. In many cases we want to
+customize existing models (e.g., adding a new feature or slightly modifying
+the behavior) instead of building new models from scratch. This modular
+framework allows to do that with minimal effort. By implementing models using
+a large number of small components that can be easily plugged in/out, we
+eliminate the need of hard-coding changes that we want to apply to a model,
+which often leads to over-complicated code and interface.
+
+The design of this tool is thus mainly focused on both fast model development
+and easy, interactive model exploration. Ultimately, this would optimize the
 iterative back-and-forth process between ideas that we have on how to model a
 particular phenomenon and insights that we get from the exploration of model
 behavior.

doc/api.rst

@@ -16,9 +16,9 @@ properties listed below. Proper use of this accessor should be like:
 
 .. code-block:: python
 
-   >>> import xarray as xr   # first import xarray
-   >>> import xsimlab       # import xsimlab (the 'xsimlab' accessor is registered)
-   >>> ds = xr.Dataset()
+   >>> import xarray as xr         # first import xarray
+   >>> import xsimlab              # import xsimlab (the 'xsimlab' accessor is registered)
+   >>> ds = xr.Dataset()           # create or load an xarray Dataset
    >>> ds.xsimlab.<meth_or_prop>   # access to the methods and properties listed below
 
 .. currentmodule:: xarray

doc/conf.py

@@ -52,6 +52,7 @@
     'sphinx.ext.extlinks',
     'sphinx.ext.napoleon',
     # 'numpydoc',
+    'sphinx.ext.todo',
     'IPython.sphinxext.ipython_directive',
     'IPython.sphinxext.ipython_console_highlighting',
     'nbsphinx',
@@ -110,7 +111,8 @@
 pygments_style = 'sphinx'
 
 # If true, `todo` and `todoList` produce output, else they produce nothing.
-todo_include_todos = False
+todo_include_todos = True
+todo_emit_warnings = True
 
 
 # -- Options for HTML output ----------------------------------------------

doc/create_model.rst

@@ -1,43 +1,10 @@
 .. _create_model:
 
-Creating models
-===============
+Create and modify models
+========================
 
-xarray-simlab's framework provides a few Python base classes, e.g.,
-:class:`~xsimlab.Variable`, :class:`~xsimlab.Process` and
-:class:`~xsimlab.Model` that can used togheter to create fully operational
-models.
+.. todo::
 
-A ``Model`` is a collection of processes that each define an interface with the
-all the "public" variables it needs, update or provide for its proper
-computation.
-
-Process
--------
-
-
-
-
-Variable
---------
-
-Generally, the ``state``, ``value``, ``rate`` and ``change`` properties should
-not be used (either get or set the value) outside of Process subclasses,
-or maybe only for debugging.
-
-Foreign Variable
-~~~~~~~~~~~~~~~~
-
-ForeignVariable.state return the same object (usually a numpy array) than
-Variable.state (replace class names by variable names in processes).
-ForeignVariable.state is actually a shortcut to ForeignVariable.ref_var.state.
-
-Variable List and Group
-~~~~~~~~~~~~~~~~~~~~~~~
-
-
-Model
------
-
-The latter class represents a model with which we interact using the xarray's
-:class:`~xarray.Dataset` structure.
+   This section has no content yet. For now, you can have a look at the
+   :doc:`examples` section that contains useful information about how to use
+   the features of xarray-simlab.

doc/faq.rst

@@ -37,6 +37,10 @@ the preliminary benchmarks that we have run show only a very small overhead.
 .. _Numba: http://numba.pydata.org/
 .. _f2py: https://docs.scipy.org/doc/numpy-dev/f2py/
 
+.. question_to_add : Does xarray-simlab allow model execution in parallel?
+
+.. question_to_add: Can xarray-simlab be used without xarray?
+
 Will xarray-simlab support Python 2.7.x?
 ----------------------------------------
 

doc/framework.rst

@@ -0,0 +1,124 @@
+.. _framework:
+
+Modeling Framework
+==================
+
+This section briefly explains the design of the xarray-simlab's modeling
+framework. For more practical details on how to create, inspect and run models,
+see the relevant sections in this user guide.
+
+Main concepts
+-------------
+
+The xarray-simlab's framework is built on only a few concepts that are layered
+onto three levels, here listed from top to bottom:
+
+- **models**, i.e., instances of :class:`~xsimlab.Model`
+- **processes**, i.e., subclasses of :class:`~xsimlab.Process`
+- **variables**, i.e., :class:`~xsimlab.Variable` objects or objects of
+  derived or related classes.
+
+A **model** is an ordered collection of processes. The role of a **process** is
+twofold: declare a given subset of the **variables** used in a model and
+define a specific set of instructions that uses, sets or modifies the values of
+these variables.
+
+Conceptually, a process is a logical component of a model. It may for example
+represent a particular physical mechanism that is described in terms of one or
+more state variables (e.g., scalar or vector fields) and one or more processes
+-- with or without parameters -- that modify those state variables through time.
+Some processes may be time-independent.
+
+.. note::
+
+   xarray-simlab does not explicitly distinguish between model parameters and
+   state variables, both are declared as variables inside their own process.
+
+.. note::
+
+   xarray-simlab does not provide any built-in logic for tasks like generating
+   computational meshes or setting boundary conditions. This should be
+   implemented in 3rd-party libraries as time-independent processes. The reason
+   is that theses tasks may vary from one model / domain to another while
+   xarray-simlab aims to be a general purpose framework.
+
+Processes are usually inter-dependent, i.e., a process often declares
+references to variables that are declared in other processes (i.e., "foreign
+variables") and sometimes even computes values for those variables. As a
+consequence, processes may not be re-usable without their dependencies.
+However, it allows declaring variables -- including useful metadata such as
+description, default values, units, etc. -- at a unique place.
+
+Simulation workflow
+-------------------
+
+A model run is divided into four stages:
+
+1. initialization
+2. run step
+3. finalize step
+4. finalization
+
+During a simulation, stages 1 and 4 are only run once while steps 2 and 3 are
+repeated for a given number of (time) steps.
+
+Each process in a model usually implements some computation for at least one of
+these stages. It is not always required for a process to perform computation at
+every stage. For example, time-independent processes don't do nothing during
+stages 2 and 3. Processes that depend on time, however, must supply an
+implementation for at least stage 2 (run step).
+
+Computation order and process dependencies
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+The order in which computations are executed during a simulation is important.
+For example, a process that sets or updates a variable must be executed before
+the execution of all other processes that need to access this variable for their
+own computation.
+
+The same order is used at every stage of a model run. It is automatically
+inferred -- as well as process dependencies -- when creating a new
+``Model`` object, using the information contained in the variables and/or
+foreign variables that are declared in each process.
+
+Process dependencies together form a Directed Acyclic Graph (DAG), which will
+eventually allow us running some processes in parallel at each stage of a model
+run (this is not yet implemented).
+
+Variable values
+---------------
+
+A single variable may accept up to 3 different values:
+
+- ``state``, i.e., the value of the variable at a given time
+- ``rate``, i.e., the value of the time-derivative at a given time
+- ``change``, i.e., the value of the time-derivative integrated for a given
+  time step.
+
+These correspond to separate properties of ``Variable`` objects.
+
+.. note::
+
+   We allow multiple values mainly to avoid creating different Variable
+   objects for the same state variable. The names and meanings given here above
+   are just conventions, though.
+
+   Model developers are free to get/set any of these properties at any stage of
+   a model run. However, it is common practice to compute the ``rate`` or
+   ``change`` values in the "run step" stage and to update the ``state`` value
+   in the "finalize step" stage.
+
+An additional property, ``value``, is also defined as an alias of ``state``.
+It should be used primarily for variables that are time-invariant. Here again it
+is just a convention as using the term "value" is a bit more accurate in this
+case.
+
+.. todo_
+
+   input variable section
+
+.. move_this_foreign_variable
+
+   ForeignVariable.state return the same object (usually a numpy array) than
+   Variable.state (replace class names by variable names in processes).
+   ForeignVariable.state is actually a shortcut to ForeignVariable.ref_var.state.

doc/index.rst

@@ -17,8 +17,6 @@ Documentation index
 * :doc:`faq`
 * :doc:`installing`
 * :doc:`examples`
-* :doc:`create_model`
-* :doc:`run_model`
 
 .. toctree::
    :maxdepth: 1
@@ -29,23 +27,38 @@ Documentation index
    faq
    installing
    examples
-   create_model
+
+**User Guide**
+
+* :doc:`framework`
+* :doc:`inspect_model`
+* :doc:`run_model`
+* :doc:`create_model`
+
+.. toctree::
+   :maxdepth: 1
+   :hidden:
+   :caption: User Guide
+
+   framework
+   inspect_model
    run_model
+   create_model
 
 **Help & reference**
 
-* :doc:`whats_new`
 * :doc:`api`
 * :doc:`develop`
+* :doc:`whats_new`
 
 .. toctree::
    :maxdepth: 1
    :hidden:
    :caption: Help & reference
 
-   whats_new
    api
    develop
+   whats_new
 
 Get in touch
 ------------
@@ -59,7 +72,7 @@ License
 
 3-clause ("Modified" or "New") BSD license.
 
-xarray-simlab is developped at the `Earth Surface Process Modelling`__ group of
-the GFZ Helmholtz Centre Potsdam.
+xarray-simlab is developed with support from the
+`Earth Surface Process Modelling`__ group of the GFZ Helmholtz Centre Potsdam.
 
 __ http://www.gfz-potsdam.de/en/section/earth-surface-process-modelling/

doc/inspect_model.rst

@@ -0,0 +1,10 @@
+.. _inspect_model:
+
+Model introspection
+===================
+
+.. todo::
+
+   This section has no content yet. For now, you can have a look at the
+   :doc:`examples` section that contains useful information about how to use
+   the features of xarray-simlab.

doc/installing.rst

@@ -53,7 +53,7 @@ For development purpose, use the following command::
 Import xarray-simlab
 --------------------
 
-To make sure that ``xarray-simlab`` is correctly installed, try import it in a
-Python console::
+To make sure that ``xarray-simlab`` is correctly installed, try import it by
+running this line::
 
-    >>> import xsimlab
+    $ python -c "import xsimlab"

doc/run_model.rst

@@ -1,9 +1,10 @@
 .. _run_model:
 
-Running models
-==============
+Run models using the xarray interface
+=====================================
 
-(TODO)
+.. todo::
 
-For now, you can have a look at the :doc:`examples` section of this documentation to
-see how xarray-simlab models can be set-up and run.
+   This section has no content yet. For now, you can have a look at the
+   :doc:`examples` section that contains useful information about how to use
+   the features of xarray-simlab.