Merge pull request #557 from SALib/faq-docs

Documentation cleanup and FAQ

docs/api.rst

@@ -75,3 +75,9 @@ High-Dimensional Model Representation
 
 .. autofunction:: SALib.analyze.hdmr.analyze
    :noindex:
+
+Regional Sensitivity Analysis
+-------------------------------------
+
+.. autofunction:: SALib.analyze.rsa.analyze
+   :noindex:

docs/index.rst

@@ -40,18 +40,38 @@ Supported Methods
   (based on `Saltelli et al. 2008 <https://dx.doi.org/10.1002/9780470725184>`__, `Pianosi et al., 2016 <https://dx.doi.org/10.1016/j.envsoft.2016.02.008>`__)
 
 
+Getting Started
+---------------
+
 .. toctree::
    :maxdepth: 2
 
-   Getting started <getting-started>
-   Basics <basics>
-   SALib Interface <basics_with_interface>
-   Advanced <advanced>
-   API <api>
+   Getting started <user_guide/getting-started>
+   Basics <user_guide/basics>
+   SALib Interface <user_guide/basics_with_interface>
+   Advanced <user_guide/advanced>
+   Wrappers <user_guide/wrappers>
+   FAQ <user_guide/faq>
+
+
+For Developers
+--------------
+
+.. toctree::
+    :maxdepth: 2
+
+    API <api>
+    Developers Guide <developers_guide>
+    Changes <changelog>
+    Complete Module Reference <api/modules>
+
+
+Other Info
+----------
+
+.. toctree::
+    :maxdepth: 2
 
    License <license>
    Authors <authors>
    Projects that use SALib <citations>
-   Developers Guide <developers_guide>
-   Changes <changelog>
-   Complete Module Reference <api/modules>

docs/basics.rst → docs/user_guide/basics.rst

@@ -329,7 +329,7 @@ Now we can extract the first-order Sobol indices for each bin of :math:`x` and p
 
     plt.show()
 
-.. figure:: assets/example_parabola.svg
+.. figure:: ../assets/example_parabola.svg
     :width: 800
     :align: center
 

docs/basics_with_interface.rst → docs/user_guide/basics_with_interface.rst

@@ -50,7 +50,7 @@ concise workflow. The sampling, evaluation and analysis trifecta can be run with
         .analyze_sobol()
     )
 
-    total_Si, first_Si, second_Si = so.to_df()
+    total_Si, first_Si, second_Si = sp.to_df()
 
     sp.plot()
     sp.heatmap()
@@ -408,7 +408,7 @@ is set to use log scale, and the figure size is adjusted with :code:`matplotlib`
 
 
 
-.. figure:: assets/example_mod_plot.svg
+.. figure:: ../assets/example_mod_plot.svg
     :width: 800
     :align: center
 
@@ -421,7 +421,7 @@ heatmap.
     sp.heatmap()
 
 
-.. figure:: assets/example_heatmap_plot.svg
+.. figure:: ../assets/example_heatmap_plot.svg
     :width: 800
     :align: center
 
@@ -546,7 +546,7 @@ Now we can extract the first-order Sobol indices for each bin of :math:`x` and p
     plt.show()
 
 
-.. figure:: assets/example_parabola.svg
+.. figure:: ../assets/example_parabola.svg
     :width: 800
     :align: center
 

docs/user_guide/faq.rst

@@ -0,0 +1,43 @@
+==========================
+Frequently Asked Questions
+==========================
+
+
+Q. How do I wrap my model?
+---------------
+
+See this guide on :doc:`wrapping models </user_guide/wrappers>`.
+
+
+Q. Which technique can I use for pre-existing results?
+------------------------------------------------------
+
+DMIM, RBD-FAST, PAWN and HDMR methods are "given-data" approaches and can be independently applied.
+
+
+Q. How do I get the sensitivity results?
+----------------------------------------
+
+Call the :code:`.to_df()` method if you would like Pandas DataFrames.
+
+If using the SALib Interface, see :doc:`SALib Interface Basics </user_guide/basics_with_interface>`.
+
+
+Q. How can I plot my results?
+-----------------------------
+
+SALib provides some basic plotting functionality. See the subsection "Basic Plotting" in :doc:`SALib Interface Basics </user_guide/basics_with_interface>`
+
+See also, `these examples <https://github.com/SALib/SALib/tree/main/examples/plotting>`_.
+
+
+Q. Why does the Sobol' method implemented in SALib normalize outputs?
+---------------------------------------------------------------------
+
+Estimates of Sobol' indices can be biased in cases where model outputs are non-centered.
+We have opted to normalize outputs with the standard deviation.
+
+See the discussion `here <https://github.com/SALib/SALib/issues/109#issuecomment-268499001>`_.
+
+In practice, non-normalized model outputs are still usable but requires larger sample sizes for
+the indices to converge.

docs/getting-started.rst → docs/user_guide/getting-started.rst

@@ -28,19 +28,20 @@ We encourage users use the latest stable version.
 Installing Prerequisite Software
 --------------------------------
 
-SALib requires `NumPy <http://www.numpy.org/>`_, `SciPy <http://www.scipy.org/>`_,
-`pandas <http://https://pandas.pydata.org/>`_,
-and `matplotlib <http://matplotlib.org/>`_ installed on your computer.  Using
-`pip <https://pip.pypa.io/en/stable/installing/>`_, these libraries can be
-installed with the following command:
+Core dependencies include:
+- `NumPy <http://www.numpy.org/>`_
+- `SciPy <http://www.scipy.org/>`_
+- `pandas <http://https://pandas.pydata.org/>`_
+- `matplotlib <http://matplotlib.org/>`_
+
+These should be installed automatically alongside SALib but otherwise they
+can be installed with the following command:
 
 ::
 
     pip install numpy scipy pandas matplotlib
 
 The packages are normally included with most Python bundles, such as Anaconda and Canopy.
-In any case, they are installed automatically when using pip to install
-SALib.
 
 
 Testing Installation

docs/user_guide/wrappers.rst

@@ -0,0 +1,191 @@
+Wrapping an existing model
+--------------------------
+
+SALib performs sensitivity analysis for any model that can be expressed in the form of :math:`f(X) = Y`,
+where :math:`X` is a matrix of inputs (often referred to as the model's factors).
+
+The analysis methods are independent of the model and can be applied non-intrusively such
+that it does not matter what :math:`f` is.
+
+Typical model implementations take the form of :math:`f(a, b, c, ...) = Y`. In other words, each model
+factor is supplied as a separate argument to the function. In such cases it is necessary to
+write a wrapper to allow use with SALib. This is illustrated here with a simple linear function:
+
+.. code:: python
+
+    def linear(a, b, x):
+        """Return y = a + b + x"""
+        return a + b + x
+
+As SALib expects a (numpy) matrix of factors, we simply "wrap" the function above like so:
+
+.. code:: python
+
+    def wrapped_linear(X, func=linear):
+        """g(X) = Y, where X := [a b x] and g(X) := f(X)"""
+        # We transpose to obtain each column (the model factors) as separate variables
+        a, b, x = X.T
+
+        # Then call the original model
+        return func(a, b, x)
+
+.. note:: Wrapped function is an argument
+    Note here that the model being "wrapped" is also passed in as an argument.
+    This will be revisited further down below.
+
+
+.. tip:: Interfacing with external models/programs
+    Here we showcase interacting with models written in Python.
+    If the model is an external program, this is where interfacing code
+    would be written.
+
+    A pragmatic approach could be to use `subprocess <https://docs.python.org/3/library/subprocess.html>`_
+    to start the external program, then read in the results.
+
+
+Constants, which SALib should not consider, can be expressed by defining default keyword arguments
+(for flexibility) or otherwise defined within the wrapper function itself.
+
+.. code:: python
+
+    def wrapped_linear_w_constant(X, a=10, func=linear):
+        """f(X, a) = Y, where X := [b x] and a = 10"""
+        # We transpose to obtain each column as separate variables
+        b, x = X.T
+
+        # Then call the original model
+        return func(a, b, x)
+
+
+Note that the first argument to the wrapper function(s) is a numpy array of shape
+:math:`N*D`, where :math:`D` is the number of model factors (dimensions) and
+:math:`N` is the number of their combinations. The argument name is, by convention,
+denoted as :code:`X`. This is to maximize compatibility with all methods provided
+in SALib as they expect the first argument to hold the model factor values.
+Using :py:func:`functools.partial` from the `functools` package to create wrappers can be useful.
+
+In this example, the model (:code:`linear()`) can be used with both scalar inputs or `numpy` arrays.
+In cases where `a`, `b` or `x` are a vector of inputs, `numpy` will automatically vectorize the
+calculation.
+
+There are many cases where the model is not (or cannot be easily) expressed in a vectorizable form.
+When using the core SALib functions directly in such cases, the user is expected to evaluate the
+model in a `for` loop themselves.
+
+.. code:: python
+
+    from SALib.sample import saltelli
+    from SALib.analyze import sobol
+
+    problem = {
+        'names': ['a', 'b', 'x'],
+        'bounds': [
+            [-1, 0],
+            [-1, 0],
+            [-1, 1],
+        ],
+        'num_vars': 3
+    }
+
+    X = saltelli.sample(problem, 64)
+    Y = np.empty(params.shape[0])
+    for i in range(params.shape[0]):
+        Y[i] = wrapped_linear(params[i, :])
+
+    res = sobol.analyze(problem, Y)
+    res.to_df()
+
+    # [         ST   ST_conf
+    #  a  0.165854  0.054096
+    #  b  0.165854  0.053200
+    #  x  0.665366  0.192756,
+    #           S1   S1_conf
+    #  a  0.167805  0.121550
+    #  b  0.167805  0.125178
+    #  x  0.665366  0.230872,
+    #                    S2   S2_conf
+    #  (a, b) -2.775558e-17  0.180493
+    #  (a, x) -3.902439e-03  0.202343
+    #  (b, x) -3.902439e-03  0.232957]
+
+This highlights one usability aspect of using the SALib `ProblemSpec` Interface - it
+automatically applies the model for each individual sample set in a `for` loop
+(at the cost of computational efficiency).
+
+.. code:: python
+
+    from SALib import ProblemSpec
+
+
+    sp = ProblemSpec({
+        'names': ['a', 'b', 'x'],
+        'bounds': [
+            [-1, 0],
+            [-1, 0],
+            [-1, 1],
+        ],
+    })
+
+    (
+        sp.sample_sobol(2**6)
+        .evaluate(wrapped_linear)
+        .analyze_sobol()
+    )
+
+    sp.to_df()
+
+    # [         ST   ST_conf
+    #  a  0.173636  0.072142
+    #  b  0.167933  0.059599
+    #  x  0.654566  0.208328,
+    #           S1   S1_conf
+    #  a  0.182788  0.111548
+    #  b  0.179003  0.145714
+    #  x  0.664727  0.241977,
+    #                S2   S2_conf
+    #  (a, b) -0.022070  0.185510
+    #  (a, x) -0.010781  0.186743
+    #  (b, x) -0.014616  0.279925]
+
+We also noted earlier that the model being "wrapped" is also passed in as an argument.
+This is to facilitate parallel evaluation, as the arguments to the wrapper
+are passed on to workers. The approach works be using Python's
+`mutable default argument <https://docs.python-guide.org/writing/gotchas/#mutable-default-arguments>`_
+behavior.
+
+Technical detail aside, defining the model this way allows the model to be evaluated in parallel:
+
+.. code:: python
+
+    from SALib import ProblemSpec
+
+
+    sp = ProblemSpec({
+        'names': ['a', 'b', 'x'],
+        'bounds': [
+            [-1, 0],
+            [-1, 0],
+            [-1, 1],
+        ],
+    })
+
+    (
+        sp.sample_sobol(2**6)
+        .evaluate(wrapped_linear, nprocs=2)
+        .analyze_sobol()
+    )
+
+    sp.to_df()
+
+    # [         ST   ST_conf
+    #  a  0.166372  0.064571
+    #  b  0.164554  0.068605
+    #  x  0.665150  0.191152,
+    #           S1   S1_conf
+    #  a  0.201450  0.152915
+    #  b  0.165128  0.124578
+    #  x  0.670300  0.254541,
+    #                S2   S2_conf
+    #  (a, b) -0.027733  0.178632
+    #  (a, x) -0.068051  0.257325
+    #  (b, x)  0.000958  0.257001]