Merge df0370c into 04bc236

.travis.yml

@@ -6,9 +6,6 @@ os:
   #- windows
 julia:
   - 1.1 #Long-term support (LTS) release: v1.0.5 (Sep 9, 2019)
-  #- 1.2
-  #- 1.3
-  #- 1.4
   - 1.5
   #- nightly
 notifications:

Project.toml

@@ -1,9 +1,8 @@
 name = "Stopping"
 uuid = "c4fe5a9e-e7fb-5c3d-89d5-7f405ab2214f"
-authors = ["Jean-Pierre Dussault <Jean-Pierre.Dussault@USherbrooke.CA>","Tangi Migot <tangi.migot@gmail.com>","Sam Goyette <samuel.goyette@usherbrooke.ca>"]
+authors = ["Jean-Pierre Dussault <Jean-Pierre.Dussault@USherbrooke.CA>", "Tangi Migot <tangi.migot@gmail.com>", "Sam Goyette <samuel.goyette@usherbrooke.ca>"]
 version = "0.2.5"
 
-
 [deps]
 DataFrames = "a93c6f00-e57d-5684-b7b6-d8193f3e46c0"
 LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
@@ -14,7 +13,6 @@ SparseArrays = "2f01184e-e22b-5df5-ae63-d93ebab69eaf"
 Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
 
 [compat]
-#NLPModels = "0.10.0, 1"
 julia = "^1.0.0"
 
 [extras]

docs/make.jl

@@ -8,6 +8,16 @@ makedocs(
     pages = [
              "Home" => "index.md",
              "API" => "api.md",
+             "Stopping's ID" => "idcard.md",
+             "State's ID" => "idcard-state.md",
+             "Meta's ID" => "idcard-stoppingmeta.md",
+             "Optimality in Stopping" => "howstopcheckoptimality.md",
+             "Stopping in action" => "example-basic-Newton.md",
+             "Stop remote control" => "idcard-stopremote.md",
+             "Stopping workflow" => "stop-workflow.md",
+             "NLPStopping" => "nlpstopping.md",
+             "LAStopping" => "lastopping.md",
+             "Readme" => "index_tuto.md",
              "Examples and tutorials" => "tutorial.md",
              "How to State" => "howtostate.md",
              "How to State for NLPs" => "howtostate-nlp.md",
@@ -30,5 +40,4 @@ makedocs(
 # See "Hosting Documentation" and deploydocs() in the Documenter manual
 # for more information.
 deploydocs(repo = "github.com/vepiteski/Stopping.jl")
-#deploydocs(repo = "github.com/Goysa2/Stopping.jl")#
 #https://juliadocs.github.io/Documenter.jl/stable/man/hosting/ ?

docs/src/example-basic-Newton.md

@@ -0,0 +1,39 @@
+## Example I: Stopping in the flow
+
+We present here a typical iterative algorithm to illustrate how to use Stopping.
+
+```julia
+function rand_solver(stp :: AbstractStopping, x0 :: AbstractVector)
+
+    x = x0
+    #First, call start! to check optimality and set an initial configuration
+    OK = update_and_start!(stp, x = x)
+
+    while !OK
+        #Run some computations and update the iterate
+        d = rand(length(x))
+        x += d
+
+        #Update the State and call the Stopping with stop!
+        OK = update_and_stop!(stp, x = x, d = d)
+    end
+
+    return stp
+end
+```
+This example shows the most basic features of Stopping. It does many checks for you. In this innocent-looking algorithm, the call to `update_and_start!` and `update_and_stop!` will verifies unboundedness of `x`, the time spent in the algorithm, the number of iterations (= number of call to `stop!`), and the domain of `x` (in case some of its components become `NaN` for instance).
+
+### FAQ: How can I remove some checks done by Stopping?
+The native instances of `AbstractStopping` available in Stopping.jl all contain an attribute `stop_remote`.
+This is a remote control for Stopping's checks.
+```julia
+typeof(stp.stop_remote) <: StopRemoteControl #stop_remote is an instance of StopRemoteControl
+```
+This attributes contains boolean values for each check done by Stopping, see
+```julia
+fieldnames(stp.stop_remote) #get all the attributes of the remote control
+```
+For instance, we can remove the unboundedness and domain check done on `x` by setting:
+```julia
+stp.stop_remote = StopRemoteControl(unbounded_and_domain_x_check = false)
+```

docs/src/howstopcheckoptimality.md

@@ -0,0 +1,37 @@
+## How Stopping checks for optimality
+
+The solver can let Stopping handles the optimality checks. We see here how it works and how to tune it in.
+
+First, the function `stop!` computes a **score** using `optimality_check` function given in the `meta`. The keywords argument given in `stop!` are passed to this function.
+```julia
+#Compute the score if !src.optimality_check
+score = stp.meta.optimality_check(stp.pb, stp.current_state; kwargs...))
+```
+The **score** is then stored in `stp.current_state.current_score`. If the **score** doesn't contain any NaN, Stopping proceeds and test whether it is within tolerances given as functions in `meta.tol_check` and `meta.tol_check_neg`.
+```julia
+#Compute the tolerances
+check_pos, check_neg = tol_check(stp.meta)
+#Test the score vs the tolerances
+optimal = _inequality_check(optimality, check_pos, check_neg)
+```
+So, overall Stopping does:
+```julia
+check_pos = stp.meta.tol_check(stp.meta.atol, stp.meta.rtol, stp.meta.optimality0)
+check_neg = stp.meta.tol_check_neg(stp.meta.atol, stp.meta.rtol, stp.meta.optimality0)
+score     = stp.meta.optimality_check(stp.pb, stp.current_state)
+check_pos ≤ score ≤ check_neg
+```
+
+### FAQ: Does it work for vector scores as well?
+
+The type of the score and tolerances are respectively initialized in the State and the Meta at the initialization of the Stopping. Hence one can use vectorized scores as long as they can be compared with the tolerances. For instance:
+- The score is a vector and tolerances are vectors of the same length or numbers.
+- The score is a tuple and tolerances are tuple or a number.
+
+### FAQ: How do I implement AND and OR conditions?
+The concatenation of two scores (AND condition) that need to be tested to zero can be represented as a vector.
+The disjunction of two score (OR condition) are represented as tuple.
+
+### FAQ: Do Stopping really computes the tolerances each time?
+
+It does unless `meta.retol` is set as `true`. This entry can be set as true from the beginning as the `tol_check` functions are evaluated once at the initialization of the `meta`.

docs/src/idcard-state.md

@@ -0,0 +1,92 @@
+## State
+
+As discussed before each Stopping contains a `current_state :: AbstractState` attribute containing the current information/state of the problem. When running the iterative loop, the `State` is updated and the `Stopping` make a decision based on this information.
+
+The `current_state` contains all the information relative to a problem. We implemented three instances as an illustration:
+- `GenericState` ;
+-  `NLPAtX` representing the state of an `NLPModel`;
+- `OneDAtX` for 1D optimization problems.
+
+`GenericState` is an illustration of the behavior of such object that minimally contains:
+- `x` the current iterate;
+- `d` the current direction;
+- `res` the current residual;
+- `current_time` the current time;
+- `current_score` the current optimality score.
+
+By convention, `x` and `current_score` are mandatory information, and the other attribute are initialized with keywords arguments:
+```julia
+GenericState(zeros(n), 0.0, d = zeros(n), current_time = NaN)
+```
+the alternative would be
+```julia
+GenericState(zeros(n), d = zeros(n), current_time = NaN)
+```
+
+Beyond the use inside Stopping, returning the State also provides the user the opportunity to use some of the information computed by the algorithm.
+
+### FAQ: Are there Type constraints when initializing a State?
+
+Yes, an AbstractState{S,T} is actually a paramtric type where `S` is the type of the `current_score` and `T` is the type of `x`.
+```julia
+x0, score0 = rand(n), Array{Float64,1}(undef, n)
+GenericState(x0, score0) #is an AbstractState{Array{Float64,1}, Array{Float64,1}}
+```
+By default, the `current_score` is a real number, hence
+```julia
+x0 = rand(n)
+GenericState(x0) #is an AbstractState{Float64, Array{Float64,1}}
+```
+These types can be obtained with the functions `xtype` and `scoretype`:
+```julia
+scoretype(stp.current_state)
+xtype(stp.current_state)
+```
+
+### FAQ: Can I design a tailored State for my problem?
+
+`NLPAtX` is an illustration of a more evolved instance associated to `NLPModels` for nonlinear optimization models. It contains:
+```julia
+mutable struct 	NLPAtX{S, T <: AbstractVector, MT <: AbstractMatrix}  <: AbstractState{S, T}
+#Unconstrained State
+    x            :: T     # current point
+    fx           :: eltype(T) # objective function
+    gx           :: T  # gradient size: x
+    Hx           :: MT  # hessian size: |x| x |x|
+#Bounds State
+    mu           :: T # Lagrange multipliers with bounds size of |x|
+#Constrained State
+    cx           :: T # vector of constraints lc <= c(x) <= uc
+    Jx           :: MT  # jacobian matrix, size: |lambda| x |x|
+    lambda       :: T    # Lagrange multipliers
+
+    d            :: T #search direction
+    res          :: T #residual
+ #Resources State
+    current_time   :: Float64
+    current_score  :: S
+    evals          :: Counters
+
+ function NLPAtX(x             :: T,
+                 lambda        :: T,
+                 current_score :: S;
+                 fx            :: eltype(T) = _init_field(eltype(T)),
+                 gx            :: T = _init_field(T),
+                 Hx            :: AbstractMatrix = _init_field(Matrix{eltype(T)}),
+                 mu            :: T = _init_field(T),
+                 cx            :: T = _init_field(T),
+                 Jx            :: AbstractMatrix = _init_field(Matrix{eltype(T)}),
+                 d             :: T = _init_field(T),
+                 res           :: T = _init_field(T),
+                 current_time  :: Float64 = NaN,
+                 evals         :: Counters = Counters()
+                 ) where {S, T <: AbstractVector}
+
+  _size_check(x, lambda, fx, gx, Hx, mu, cx, Jx)
+
+  return new{S, T, Matrix{eltype(T)}}(x, fx, gx, Hx, mu, cx, Jx, lambda, d, 
+                                      res, current_time, current_score, evals)
+ end
+end
+```
+`_init_field(T)` initializes a value for a given type guaranteing type stability and minimal storage.

docs/src/idcard-stoppingmeta.md

@@ -0,0 +1,52 @@
+## Stopping's attributes ID: StoppingMeta
+
+Usual instances of `AbstractStopping` contains a `StoppingMeta <: <: AbstractStoppingMeta` (`stp.meta`), which controls the various tolerances and thresholds used by the functions `start!` and `stop!`.
+- `atol                :: Number   = 1.0e-6`
+- `rtol                :: Number   = 1.0e-15`
+- `optimality0         :: Number   = 1.0`
+- `tol_check           :: Function = (atol :: Number, rtol :: Number, opt0 :: Number) -> max(atol,rtol*opt0)`
+- `tol_check_neg       :: Function = (atol :: Number, rtol :: Number, opt0 :: Number) -> - tol_check(atol,rtol,opt0)`
+- `optimality_check    :: Function = (a,b) -> Inf`
+- `retol               :: Bool     = true`
+- `unbounded_threshold :: Number   = 1.0e50, #typemax(Float64)`
+- `unbounded_x         :: Number   = 1.0e50`
+- `max_f               :: Int      = typemax(Int)`
+- `max_cntrs           :: Dict{Symbol,Int} = Dict{Symbol,Int64}()`
+- `max_eval            :: Int      = 20000`
+- `max_iter            :: Int      = 5000`
+- `max_time            :: Float64  = 300.0`
+- `start_time          :: Float64  = NaN`
+- `meta_user_struct    :: Any      = nothing`
+- `user_check_func!    :: Function = (stp :: AbstractStopping, start :: Bool) -> nothing`
+The default constructor for the meta uses above values, and they can all be modified using keywords
+```julia
+meta = StoppingMeta(rtol = 0.0) #will set `rtol` as 0.0.
+```
+
+`StoppingMeta` also contains the various status related to the checks:
+```julia
+OK_check(meta) #returns true if one of the check is true.
+```
+
+### FAQ: Are there Type constraints when initializing a StoppingMeta?
+
+An `StoppingMeta{TolType, CheckType, MUS, IntType}` is actually a paramtric type:
+```julia
+checktype(meta) #CheckType: return type of `tol_check` and `tol_check_neg` 
+toltype(meta) #TolType: type of the tolerances
+metausertype(meta) #MUS: type of the user-defined structure
+inttype(meta) #IntType: type of integer tolerances
+```
+
+### FAQ: What is `user_check_func!`?
+
+This is a callback function called in the execution of the function `stop!` or `start!`. This function takes two input `stp <: AbstractStopping` and a boolean set as `true` if called from `start!` and `false` if called from `stop!`. To eventually returns a stopping status, the function has to update `stp.meta.stopbyuser`.
+
+For instance, if one want to stop when $$\log(x) < 1$$ in `stop!`:
+```julia
+function test(stp, start)
+  stp.meta.stopbyuser = !start && (log(stp.current_state.x) < 1)
+end
+user_check_func! = test
+```
+The exclamation mark (!) is a naming convention used when the function modifies input.

docs/src/idcard-stopremote.md

@@ -0,0 +1,37 @@
+## Stopping's attributes ID: StopRemoteControl
+
+Usual instances of `AbstractStopping` contains a `StopRemoteControl <: AbstractStopRemoteControl` (`stp.stop_remote`), which controls the various checks run by the functions `start!` and `stop!`. An instance of `StopRemoteControl` contains:
+- `unbounded_and_domain_x_check :: Bool`
+- `domain_check                 :: Bool`
+- `optimality_check             :: Bool`
+- `infeasibility_check          :: Bool`
+- `unbounded_problem_check      :: Bool`
+- `tired_check                  :: Bool`
+- `resources_check              :: Bool`
+- `stalled_check                :: Bool`
+- `iteration_check              :: Bool`
+- `main_pb_check                :: Bool`
+- `user_check                   :: Bool`
+- `user_start_check             :: Bool`
+- `cheap_check                  :: Bool`
+Only the last attributes, `cheap_check`, is not related with a specific check. Set as `true`, it stopped whenever one of the checks is successful and the algorithm needs to stop. It is `false` by default. All the other entries are set as `true` by default, i.e.
+```julia
+#initializes a remote control with all the checks on.
+src = StopRemoteControl()
+```
+In order to remove some checks, it suffices to use keywords:
+```julia
+#remove time and iteration checks.
+src = StopRemoteControl(tired_check = false, iteration_check = false)
+```
+
+### FAQ: Is there performance issues with all these checks?
+Assuming that `x` is a vector of length `n`, some of these checks are indeed in O(n), which can be undesirable for some applications. In this case, you can either initialize a "cheap" remote control as follows
+```julia
+#initialize a StopRemoteControl with 0(n) checks set as false
+src = cheap_stop_remote_control()
+```
+or deactivate the tests by hand as shown previously.
+
+### FAQ: How can I fine-tune these checks?
+All these checks can be fine-tuned by selecting entries in the `StoppingMeta`.

docs/src/idcard.md

@@ -0,0 +1,39 @@
+## Stopping
+
+A Stopping is an instance (a subtype) of an `AbstractStopping`. Such instances minimally contain:
+-  `problem :: Any` an arbitrary instance of a problem;
+-  `meta :: AbstractStoppingMeta` contains the used parameters and stopping statuses;
+-  `current_state :: AbstractState` current information/state of the problem.
+
+While the `problem` is up to the user, the `meta` and the `current_state` are specific features of Stopping.jl.
+The `meta` contains all the parameters relative to the stopping criteria (tolerances, limits ...). We implemented
+`StoppingMeta()` which offers a set of default parameters that can be easily modified with keyword arguments. See [StoppingMeta](https://github.com/vepiteski/Stopping.jl/blob/master/src/Stopping/StoppingMetamod.jl) for more detailed information. The native instances of `AbstractStopping` (`GenericStopping`, `NLPStoppping`, etc) contains more attributes (`stop_remote`, `main_stp`, `listofstates`, `stopping_user_struct`) that we will developed later on.
+
+The `current_state` contains all the information relative to a problem. We implemented a `GenericState` as an
+illustration of the behavior of such object that typically contains:
+- `x` the current iterate;
+- `d` the current direction;
+- `res` the current residual;
+- `current_time` the current time;
+- `current_score` the current optimality score;
+- ... other information relative to the problems.
+
+When running the iterative loop, the `State` is updated and the `Stopping` make a decision based on this information.
+
+## Main Methods
+
+Stopping's main behavior is represented by two functions:
+* `start!(:: AbstractStopping)` initializes the time and the tolerance at the starting point and stopping criteria.
+* `stop!(:: AbstractStopping)` checks stopping criteria
+
+Stopping uses the information furnished by the State to make a decision. Communication between the two can be done through the following functions:
+* `update_and_start!(stp :: AbstractStopping; kwargs...)` updates the states with information furnished as kwargs and then call start!.
+* `update_and_stop!(stp :: AbstractStopping; kwargs...)` updates the states with information furnished as kwargs and then call stop!.
+* `fill_in!(stp :: AbstractStopping, x :: xtype(stp.current_state))` a function that fills in all the State with all the information required to evaluate the stopping functions correctly. This can reveal useful, for instance, if the user do not trust the information furnished by the algorithm in the State.
+* `reinit!(stp :: AbstractStopping)` reinitialize the entries of the Stopping to reuse for another call.
+
+### FAQ: How do I get more information?
+As usual in Julia, we can use `?` to get functions' documentation.
+```julia
+? Stopping.stop!
+```