updated docs: bound constraints

TODO

@@ -89,6 +89,7 @@
 * "anonymous" function types: %f = ~1/(1+exp(11*(~0.663-x)))  -> Anon0(a0,a1)
 
 * fitting - external libraries to be considered/tested:
+  - ceres-solver (or http://dlib.net/optimization.html)
   - libgencurvefit (Differential Evolution),
   - PaGMO library that collects many optimisation methods.
 

doc/fit.rst

@@ -61,6 +61,8 @@ is superior.
 Uncertainty of Parameters
 -------------------------
 
+(It is easier for me to find a quote than to express it myself).
+
 From the book J. Wolberg, *Data Analysis Using the Method of Least Squares: Extracting the Most Information from Experiments*, Springer, 2006, p.50:
 
    (...) we turn to the task of determining the uncertainties associated
@@ -84,7 +86,8 @@ Terms of the *C* matrix are given as (p. 47 in the same book):
   C_{jk}=\sum_{i=1}^n w_i \frac{\partial f}{\partial a_j} \frac{\partial f}{\partial a_k}
 
 :math:`\sigma_{a_k}` above is often called a *standard error*.
-Having standard errors, it is easy to calculate confidence intervals.
+Having standard errors, it is easy to calculate *confidence intervals*.
+But all these values should be used with care.
 Now another book will be cited: H. Motulsky and A. Christopoulos,
 *Fitting Models to Biological Data Using Linear and Nonlinear Regression:
 A Practical Guide to Curve Fitting*, Oxford University Press, 2004.
@@ -109,6 +112,44 @@ __ http://www.graphpad.com/manuals/prism4/RegressionBook.pdf
    reported standard error and confidence intervals won’t be helpful.
 
 
+.. _bound_constraints:
+
+Bound Constraints
+-----------------
+
+*Simple-variables* can have a :ref:`domain <domain>`.
+Fitting method ``mpfit`` (Lev-Mar implementation) and the methods
+from the NLOpt library use domains to constrain the parameters
+-- they never let the parameters go outside of the domain during fitting.
+
+In the literature, bound constraints are also called box constraints or,
+more generally, inequality constraints.
+Now a quotation discouraging the use of constraints. Peter Gans,
+*Data Fitting in the Chemical Sciences by the Method of Least Squares*,
+John Wiley & Sons, 1992, chapter 5.2.2:
+
+   Before looking at ways of dealing with inequality constraints we must ask a
+   fundamental question: are they necessary? In the physical sciences and in
+   least-squares minimizations in particular, inequality constraints are not
+   always justified. The most common inequality constraint is that some number
+   that relates to a physical quantity should be positive, *p*:sub:`j` > 0. If
+   an unconstrained minimalization leads to a negative value, what are we to
+   conclude? There are three possibilities; (a) the refinement has converged
+   to a false minimum; (b) the model is wrong; (c) the parameter is not well
+   defined by the data and is not significantly different from zero. In each
+   of these three cases a remedy is at hand that does not involve constrained
+   minimization: (a) start the refinement from good first estimates of the
+   parameters; (b) change the model; (c) improve the quality of the data by
+   further experimental work. If none of these remedies cure the problem of
+   non-negativity constraints, then something is seriously wrong with the
+   patient, and constrained minimization will probably not help.
+
+Setting the domain is described in the section :ref:`domain`.
+
+For a convenience, the :option:`box_constraints` option can globally disable
+(and re-enable) the constraints.
+
+
 .. _fitting_cmd:
 
 Fitting Related Commands

doc/model.rst

@@ -159,26 +159,39 @@ Variables can be deleted using the command::
 
    delete $variable
 
+
 .. _domain:
 
-Some fitting algorithms randomize the parameters of the model
-(i.e. they randomize simple variables). To effectively use such algorithms,
-the user should specify a :dfn:`domain` for each simple-variable,
-i.e. the minimum and maximum value.
-The domain does not imply any constraints on the value
-the variable can have -- it is only a hint for fitting algorithms.
+Domains
+-------
+
+Simple-variables may have a :dfn:`domain`,
+which is used for two things when fitting.
 
-The default algorithm (Lev-Mar) does not need it, so in most cases you
-do not need to worry about domains.
+Most importantly, fitting methods that support bound constraints
+use the domain as lower and/or upper bounds.
+See the section :ref:`bound_constraints` for details.
+
+The other use is for randomizing parameters (simple-variables) of the model.
+Methods that stochastically initialize or modify parameters
+(usually generating a set of initial points) need well-defined
+domains (minimum and maximum values for parameters) to work effectively.
+Such methods include Nelder-Mead simplex and Genetic Algorithms,
+but not the default Lev-Mar method, so in most cases you
+do not need to worry about it.
 
-Domains are used by the Nelder-Mead method and Genetic Algorithms.
 The syntax is as follows::
 
     $a = ~12.3 [0:20] # initial values are drawn from the (0, 20) range
-
-If the domain is not specified, the default domain is used, which is
-±\ *p*\ % of the current value, where *p* can be set using the
-:option:`domain_percent` option.
+    $a = ~12.3 [0:]   # only lower bound
+    $a = ~12.3 [:20]  # only upper bound
+    $a = ~15.0        # domain stays the same
+    $a = ~15.0 []     # no domain
+    $a = ~{$a} [0:20] # domain is set again
+
+If the domain is not specified but it is required (for the latter use)
+by the fitting method, we assume it to be ±\ *p*\ % of the current value,
+where *p* can be set using the :option:`domain_percent` option.
 
 Function Types and Functions
 ----------------------------