From 8c9030c329933eaa72aa30b162fd75f6300a49c8 Mon Sep 17 00:00:00 2001 From: fpacaud Date: Wed, 19 Jun 2019 22:40:25 +0200 Subject: [PATCH 1/6] Add docstring for CachingOptimizer in apireference.md --- docs/src/apireference.md | 11 +++++++++++ src/Utilities/cachingoptimizer.jl | 12 ++++++------ 2 files changed, 17 insertions(+), 6 deletions(-) diff --git a/docs/src/apireference.md b/docs/src/apireference.md index 15837b3093..38ebedd87b 100644 --- a/docs/src/apireference.md +++ b/docs/src/apireference.md @@ -546,3 +546,14 @@ Utilities.load_variables Utilities.load Utilities.load_constraint ``` + +### Caching optimizer + +Add some text here. + +```@docs +Utilities.CachingOptimizer +Utilities.attach_optimizer +Utilities.reset_optimizer +Utilities.drop_optimizer +``` diff --git a/src/Utilities/cachingoptimizer.jl b/src/Utilities/cachingoptimizer.jl index 919bf0e75d..6aafc04574 100644 --- a/src/Utilities/cachingoptimizer.jl +++ b/src/Utilities/cachingoptimizer.jl @@ -14,11 +14,11 @@ A `CachingOptimizer` may be in one of three possible states (`CachingOptimizerSt - `NO_OPTIMIZER`: The CachingOptimizer does not have any optimizer. - `EMPTY_OPTIMIZER`: The CachingOptimizer an empty optimizer. The optimizer is not synchronized with the cached model. -- `ATTACHED_OPTIMIZER`: The CachingOptimizer a optimizer, and it is synchronized with the cached model. +- `ATTACHED_OPTIMIZER`: The CachingOptimizer an optimizer, and it is synchronized with the cached model. A `CachingOptimizer` has two modes of operation (`CachingOptimizerMode`): -- `MANUAL`: The only methods that change the state of the `CachingOptimizer` are [`reset_optimizer`](@ref), [`drop_optimizer`](@ref), and [`attach_optimizer`](@ref). Attempting to perform an operation in the incorrect state results in an error. +- `MANUAL`: The only methods that change the state of the `CachingOptimizer` are [`Utilities.reset_optimizer`](@ref), [`Utilities.drop_optimizer`](@ref), and [`Utilities.attach_optimizer`](@ref). Attempting to perform an operation in the incorrect state results in an error. - `AUTOMATIC`: The `CachingOptimizer` changes its state when necessary. For example, `optimize!` will automatically call `attach_optimizer` (an optimizer must have been previously set). Attempting to add a constraint or perform a modification not supported by the optimizer results in a drop to `EMPTY_OPTIMIZER` mode. """ mutable struct CachingOptimizer{OptimizerType, ModelType<:MOI.ModelLike} <: MOI.AbstractOptimizer @@ -41,7 +41,7 @@ end CachingOptimizer(model_cache::MOI.ModelLike, optimizer::AbstractOptimizer) Creates an `CachingOptimizer` in `AUTOMATIC` mode, with the optimizer `optimizer`. -The model_cache manager returned behaves like an `AbstractOptimizer` as long as no +The `model_cache` manager returned behaves like an `AbstractOptimizer` as long as no `CachingOptimizer`-specific functions (e.g. `reset_optimizer`) are called on it. The type of the optimizer returned is `CachingOptimizer{typeof(optimizer), typeof(model_cache)}` so it does not support the function @@ -59,14 +59,14 @@ end """ state(m::CachingOptimizer)::CachingOptimizerState -Returns the state of the CachingOptimizer `m`. See [`CachingOptimizer`](@ref). +Returns the state of the CachingOptimizer `m`. See [`Utilities.CachingOptimizer`](@ref). """ state(m::CachingOptimizer) = m.state """ mode(m::CachingOptimizer)::CachingOptimizerMode -Returns the operating mode of the CachingOptimizer `m`. See [`CachingOptimizer`](@ref). +Returns the operating mode of the CachingOptimizer `m`. See [`Utilities.CachingOptimizer`](@ref). """ mode(m::CachingOptimizer) = m.mode @@ -121,7 +121,7 @@ end Attaches the optimizer to `model`, copying all model data into it. Can be called only from the `EMPTY_OPTIMIZER` state. If the copy succeeds, the `CachingOptimizer` will be in state `ATTACHED_OPTIMIZER` after the call, -otherwise an error is thrown; see [`copy_to`](@ref) for more details on which +otherwise an error is thrown; see [`MathOptInterface.copy_to`](@ref) for more details on which errors can be thrown. """ function attach_optimizer(model::CachingOptimizer) From 407079f87828b33d6b31d13c9b37a077f6bc3cfd Mon Sep 17 00:00:00 2001 From: fpacaud Date: Sun, 23 Jun 2019 22:55:36 +0200 Subject: [PATCH 2/6] Add instruction to use CachingOptimizer --- docs/src/apireference.md | 28 +++++++++++++++++++++++++++- src/Utilities/cachingoptimizer.jl | 4 ++-- 2 files changed, 29 insertions(+), 3 deletions(-) diff --git a/docs/src/apireference.md b/docs/src/apireference.md index 38ebedd87b..d752ee91ac 100644 --- a/docs/src/apireference.md +++ b/docs/src/apireference.md @@ -549,11 +549,37 @@ Utilities.load_constraint ### Caching optimizer -Add some text here. +Some solvers do not support incremental definition of optimization +models. Nevertheless, you are still able to build incrementally an optimization +model with such solvers. MathOptInterface allows to use a utility, +[`Utilities.CachingOptimizer`](@ref), that will store in a [`ModelLike`](@ref) `model_cache` +the optimization model during its incremental definition. Once the +model completely defined, the `CachingOptimizer` specifies all problem's +information to the underlying solver, all in once. + +The way to operate a `CachingOptimizer` is as follow. +1) Suppose you want to cache a (empty) [`AbstractOptimizer`](@ref) `optimizer`. +2) Define a [`ModelLike`](@ref) `model_cache` that will store the incremental definition of the model. +3) Cache `optimizer` with `model_cache` by calling `CachingOptimizer(model_cache, optimizer)`. This method returns an empty `AbstractOptimizer` `cached_optimizer`. +4) Define your optimization model as usual by using `cached_optimizer` instead of `optimizer`. This will allocate the model directly to `model_cache` instead of `optimizer`. For instance, to add two variables in the model: +```julia +MOI.add_variables(cached_optimizer, 2) +``` +5) Once the model properly defined, call [`Utilities.attach_optimizer`](@ref) to copy the optimization model all in once inside `optimizer`. This method uses the MathOptInterface method [`copy_to`](@ref) to copy information from `model_cache` to `optimizer`. + +Note that: +- [`Utilities.drop_optimizer`](@ref) drops the underlying `optimizer` from `cached_optimizer`, without emptying it. +- [`Utilities.reset_optimizer`](@ref) empties `optimizer` inside `cached_optimizer`, without droping it. + +Bridging correctly a solver with a `CachingOptimizer` requires to +implement properly the [Allocate-Load API](@ref) inside the solver's +MathOptInterface wrapper. ```@docs Utilities.CachingOptimizer Utilities.attach_optimizer Utilities.reset_optimizer Utilities.drop_optimizer +Utilities.state +Utilities.mode ``` diff --git a/src/Utilities/cachingoptimizer.jl b/src/Utilities/cachingoptimizer.jl index 6aafc04574..05194675a7 100644 --- a/src/Utilities/cachingoptimizer.jl +++ b/src/Utilities/cachingoptimizer.jl @@ -13,8 +13,8 @@ construction and modification even when the optimizer doesn't. A `CachingOptimizer` may be in one of three possible states (`CachingOptimizerState`): - `NO_OPTIMIZER`: The CachingOptimizer does not have any optimizer. -- `EMPTY_OPTIMIZER`: The CachingOptimizer an empty optimizer. The optimizer is not synchronized with the cached model. -- `ATTACHED_OPTIMIZER`: The CachingOptimizer an optimizer, and it is synchronized with the cached model. +- `EMPTY_OPTIMIZER`: The CachingOptimizer has an empty optimizer. The optimizer is not synchronized with the cached model. +- `ATTACHED_OPTIMIZER`: The CachingOptimizer has an optimizer, and it is synchronized with the cached model. A `CachingOptimizer` has two modes of operation (`CachingOptimizerMode`): From 680ea2307801998a19a95a51817ea911e4481014 Mon Sep 17 00:00:00 2001 From: fpacaud Date: Mon, 1 Jul 2019 21:55:51 +0200 Subject: [PATCH 3/6] Modify CachingOptimizer documentation --- docs/src/apireference.md | 54 ++++++++++++++++++++----------- src/Utilities/cachingoptimizer.jl | 23 +++++++++---- 2 files changed, 53 insertions(+), 24 deletions(-) diff --git a/docs/src/apireference.md b/docs/src/apireference.md index 77ab214ac1..44faa83b15 100644 --- a/docs/src/apireference.md +++ b/docs/src/apireference.md @@ -554,26 +554,44 @@ models. Nevertheless, you are still able to build incrementally an optimization model with such solvers. MathOptInterface allows to use a utility, [`Utilities.CachingOptimizer`](@ref), that will store in a [`ModelLike`](@ref) `model_cache` the optimization model during its incremental definition. Once the -model completely defined, the `CachingOptimizer` specifies all problem's +model is completely defined, the `CachingOptimizer` specifies all problem's information to the underlying solver, all in once. -The way to operate a `CachingOptimizer` is as follow. -1) Suppose you want to cache a (empty) [`AbstractOptimizer`](@ref) `optimizer`. -2) Define a [`ModelLike`](@ref) `model_cache` that will store the incremental definition of the model. -3) Cache `optimizer` with `model_cache` by calling `CachingOptimizer(model_cache, optimizer)`. This method returns an empty `AbstractOptimizer` `cached_optimizer`. -4) Define your optimization model as usual by using `cached_optimizer` instead of `optimizer`. This will allocate the model directly to `model_cache` instead of `optimizer`. For instance, to add two variables in the model: -```julia -MOI.add_variables(cached_optimizer, 2) -``` -5) Once the model properly defined, call [`Utilities.attach_optimizer`](@ref) to copy the optimization model all in once inside `optimizer`. This method uses the MathOptInterface method [`copy_to`](@ref) to copy information from `model_cache` to `optimizer`. - -Note that: -- [`Utilities.drop_optimizer`](@ref) drops the underlying `optimizer` from `cached_optimizer`, without emptying it. -- [`Utilities.reset_optimizer`](@ref) empties `optimizer` inside `cached_optimizer`, without droping it. - -Bridging correctly a solver with a `CachingOptimizer` requires to -implement properly the [Allocate-Load API](@ref) inside the solver's -MathOptInterface wrapper. +The function [`Utilities.state`](@ref) allows to query the state +of the optimizer cached inside a `CachingOptimizer`. The state +could be: +* `NO_OPTIMIZER`, if no optimizer is attached; +* `EMPTY_OPTIMIZER`, if the attached optimizer is empty; +* `ATTACHED_OPTIMIZER`, if the attached optimizer is synchronized with the + cached model defined in `CachingOptimizer`. + +Note that there exists methods to modify the attached optimizer: +* [`Utilities.attach_optimizer`](@ref) attachs a new `optimizer` + to a `cached_optimizer` with state `EMPTY_OPTIMIZER`. + The state of `cached_optimizer` is set to `ATTACHED_OPTIMIZER` after the call. +* [`Utilities.drop_optimizer`](@ref) drops the underlying `optimizer` + from `cached_optimizer`, without emptying it. The state of `cached_optimizer` + is set to `NO_OPTIMIZER` after the call. +* [`Utilities.reset_optimizer`](@ref) empties `optimizer` inside + `cached_optimizer`, without droping it. The state of `cached_optimizer` + is set to `EMPTY_OPTIMIZER` after the call. + +The way to operate a `CachingOptimizer` depends whether the mode +is set to `AUTOMATIC` or to `MANUAL`. +* In `MANUAL` mode, the state of the `CachingOptimizer` changes only + if the methods [`Utilities.attach_optimizer`](@ref), + [`Utilities.reset_optimizer`](@ref) or [`Utilities.drop_optimizer`](@ref) + are being called. Any unattended operation results in an error. +* In `AUTOMATIC` mode, the state of the `CachingOptimizer` changes when + necessary. Any modification not supported by the solver (e.g. dropping + a constraint) results in a drop to the state `EMPTY_OPTIMIZER`. + +Wrapping correctly a solver with a `CachingOptimizer` requires to +implement properly the [Allocate-Load API](@ref) inside the +MathOptInterface wrapper of the solver. +Note that bridges do not yet implement the Allocate-Load API. A `CachingOptimizer` +is required if you are using a bridged solver and want to use the +Allocate-Load API, as explained in the section [Implementing copy](@ref). ```@docs Utilities.CachingOptimizer diff --git a/src/Utilities/cachingoptimizer.jl b/src/Utilities/cachingoptimizer.jl index 05194675a7..4d32e93cd2 100644 --- a/src/Utilities/cachingoptimizer.jl +++ b/src/Utilities/cachingoptimizer.jl @@ -12,14 +12,22 @@ construction and modification even when the optimizer doesn't. A `CachingOptimizer` may be in one of three possible states (`CachingOptimizerState`): -- `NO_OPTIMIZER`: The CachingOptimizer does not have any optimizer. -- `EMPTY_OPTIMIZER`: The CachingOptimizer has an empty optimizer. The optimizer is not synchronized with the cached model. -- `ATTACHED_OPTIMIZER`: The CachingOptimizer has an optimizer, and it is synchronized with the cached model. +* `NO_OPTIMIZER`: The CachingOptimizer does not have any optimizer. +* `EMPTY_OPTIMIZER`: The CachingOptimizer has an empty optimizer. + The optimizer is not synchronized with the cached model. +* `ATTACHED_OPTIMIZER`: The CachingOptimizer has an optimizer, and it is synchronized with the cached model. A `CachingOptimizer` has two modes of operation (`CachingOptimizerMode`): -- `MANUAL`: The only methods that change the state of the `CachingOptimizer` are [`Utilities.reset_optimizer`](@ref), [`Utilities.drop_optimizer`](@ref), and [`Utilities.attach_optimizer`](@ref). Attempting to perform an operation in the incorrect state results in an error. -- `AUTOMATIC`: The `CachingOptimizer` changes its state when necessary. For example, `optimize!` will automatically call `attach_optimizer` (an optimizer must have been previously set). Attempting to add a constraint or perform a modification not supported by the optimizer results in a drop to `EMPTY_OPTIMIZER` mode. +* `MANUAL`: The only methods that change the state of the `CachingOptimizer` + are [`Utilities.reset_optimizer`](@ref), [`Utilities.drop_optimizer`](@ref), + and [`Utilities.attach_optimizer`](@ref). + Attempting to perform an operation in the incorrect state results in an error. +* `AUTOMATIC`: The `CachingOptimizer` changes its state when necessary. For + example, `optimize!` will automatically call `attach_optimizer` (an + optimizer must have been previously set). Attempting to add a constraint or + perform a modification not supported by the optimizer results in a drop to + `EMPTY_OPTIMIZER` mode. """ mutable struct CachingOptimizer{OptimizerType, ModelType<:MOI.ModelLike} <: MOI.AbstractOptimizer optimizer::Union{Nothing, OptimizerType} @@ -387,7 +395,10 @@ function MOI.set(m::CachingOptimizer, attr::MOI.AbstractModelAttribute, value) MOI.set(m.model_cache, attr, value) end -function MOI.set(m::CachingOptimizer, attr::Union{MOI.AbstractVariableAttribute,MOI.AbstractConstraintAttribute}, index::MOI.Index, value) +function MOI.set(m::CachingOptimizer, + attr::Union{MOI.AbstractVariableAttribute, + MOI.AbstractConstraintAttribute}, + index::MOI.Index, value) if m.state == ATTACHED_OPTIMIZER optimizer_index = m.model_to_optimizer_map[index] optimizer_value = attribute_value_map(m.model_to_optimizer_map, value) From 87905577a71544e3f859748dd7d996124beba9d4 Mon Sep 17 00:00:00 2001 From: fpacaud Date: Mon, 1 Jul 2019 21:59:52 +0200 Subject: [PATCH 4/6] Fix misformatted markdown in Benchmarks section --- src/Benchmarks/Benchmarks.jl | 1 - 1 file changed, 1 deletion(-) diff --git a/src/Benchmarks/Benchmarks.jl b/src/Benchmarks/Benchmarks.jl index f126106320..9efa017405 100644 --- a/src/Benchmarks/Benchmarks.jl +++ b/src/Benchmarks/Benchmarks.jl @@ -22,7 +22,6 @@ Use `exclude` to exclude a subset of benchmarks. suite() do GLPK.Optimizer() end - suite(exclude = [r"delete"]) do Gurobi.Optimizer(OutputFlag=0) end From 28f4a7b7d29b4001cc7e4a4a516b941277939ca9 Mon Sep 17 00:00:00 2001 From: fpacaud Date: Tue, 2 Jul 2019 22:19:00 +0200 Subject: [PATCH 5/6] Refact documentation in PR#777 --- docs/src/apimanual.md | 6 +++--- docs/src/apireference.md | 11 ++++------- 2 files changed, 7 insertions(+), 10 deletions(-) diff --git a/docs/src/apimanual.md b/docs/src/apimanual.md index dffff62af1..8d715ed347 100644 --- a/docs/src/apimanual.md +++ b/docs/src/apimanual.md @@ -919,9 +919,9 @@ MOI.set(model, MyPackage.PrintLevel(), 0) ### Implementing copy Avoid storing extra copies of the problem when possible. This means that solver -wrappers should not use `CachingOptimizer` as part of the wrapper. Instead, do -one of the following to load the problem (assuming the solver wrapper type is -called `Optimizer`): +wrappers should not use [`Utilities.CachingOptimizer`](@ref) as part of the wrapper. +Instead, do one of the following to load the problem (assuming the solver +wrapper type is called `Optimizer`): * If the solver supports loading the problem incrementally, implement [`add_variable`](@ref), [`add_constraint`](@ref) for supported constraints and diff --git a/docs/src/apireference.md b/docs/src/apireference.md index 44faa83b15..0766e1942e 100644 --- a/docs/src/apireference.md +++ b/docs/src/apireference.md @@ -581,17 +581,14 @@ is set to `AUTOMATIC` or to `MANUAL`. * In `MANUAL` mode, the state of the `CachingOptimizer` changes only if the methods [`Utilities.attach_optimizer`](@ref), [`Utilities.reset_optimizer`](@ref) or [`Utilities.drop_optimizer`](@ref) - are being called. Any unattended operation results in an error. + are being called. Any unattended operation results in an error. * In `AUTOMATIC` mode, the state of the `CachingOptimizer` changes when necessary. Any modification not supported by the solver (e.g. dropping a constraint) results in a drop to the state `EMPTY_OPTIMIZER`. -Wrapping correctly a solver with a `CachingOptimizer` requires to -implement properly the [Allocate-Load API](@ref) inside the -MathOptInterface wrapper of the solver. -Note that bridges do not yet implement the Allocate-Load API. A `CachingOptimizer` -is required if you are using a bridged solver and want to use the -Allocate-Load API, as explained in the section [Implementing copy](@ref). +When calling [`Utilities.attach_optimizer`](@ref), the `CachingOptimizer` copies +the cached model to the optimizer with [`MathOptInterface.copy_to`](@ref). +We refer to [Implementing copy](@ref) for more details. ```@docs Utilities.CachingOptimizer From 142f86d67b5b0e6e4a4d5b32c8cf2cb93690b822 Mon Sep 17 00:00:00 2001 From: fpacaud Date: Mon, 8 Jul 2019 09:03:12 +0200 Subject: [PATCH 6/6] Adress comments --- docs/src/apireference.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/src/apireference.md b/docs/src/apireference.md index 0766e1942e..47729a37c2 100644 --- a/docs/src/apireference.md +++ b/docs/src/apireference.md @@ -551,11 +551,11 @@ Utilities.load_constraint Some solvers do not support incremental definition of optimization models. Nevertheless, you are still able to build incrementally an optimization -model with such solvers. MathOptInterface allows to use a utility, -[`Utilities.CachingOptimizer`](@ref), that will store in a [`ModelLike`](@ref) `model_cache` +model with such solvers. MathOptInterface provides a utility, +[`Utilities.CachingOptimizer`](@ref), that will store in a [`ModelLike`](@ref) the optimization model during its incremental definition. Once the -model is completely defined, the `CachingOptimizer` specifies all problem's -information to the underlying solver, all in once. +model is completely defined, the `CachingOptimizer` specifies all problem +information to the underlying solver, all at once. The function [`Utilities.state`](@ref) allows to query the state of the optimizer cached inside a `CachingOptimizer`. The state @@ -565,7 +565,7 @@ could be: * `ATTACHED_OPTIMIZER`, if the attached optimizer is synchronized with the cached model defined in `CachingOptimizer`. -Note that there exists methods to modify the attached optimizer: +The following methods modify the state of the attached optimizer: * [`Utilities.attach_optimizer`](@ref) attachs a new `optimizer` to a `cached_optimizer` with state `EMPTY_OPTIMIZER`. The state of `cached_optimizer` is set to `ATTACHED_OPTIMIZER` after the call.