Unconstrained Optimization

[FFroehlich]

Works with all optimizers but pyswarm and dlib. Also some optimizers are terribad ...

[codecov-io]

Codecov Report

Merging #519 (e1a9cea) into develop (1b049af) will increase coverage by 0.07%.
The diff coverage is 93.93%.

@@             Coverage Diff             @@
##           develop     #519      +/-   ##
===========================================
+ Coverage    90.14%   90.22%   +0.07%     
===========================================
  Files           72       72              
  Lines         4291     4315      +24     
===========================================
+ Hits          3868     3893      +25     
+ Misses         423      422       -1     

Impacted Files	Coverage Δ
pypesto/optimize/task.py	100.00% <ø> (ø)
pypesto/visualize/optimizer_history.py	93.04% <0.00%> (ø)
pypesto/optimize/optimizer.py	87.26% <85.71%> (-0.07%)	⬇️
pypesto/optimize/optimize.py	98.24% <100.00%> (+0.74%)	⬆️
pypesto/optimize/options.py	100.00% <100.00%> (ø)
pypesto/engine/multi_process.py	92.30% <0.00%> (+7.69%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1b049af...e1a9cea. Read the comment docs.

[yannikschaelte]

I believe this should all work fine, no question. However, I was wondering conceptually whether it may be desirable to make the distinction clearer between startpoint and objective bounds, same as it is done in petab (the difference is not supported yet), e.g. by not using the "ConstraintRemover" but instead allowing to pass both ub and ub_init to the problem class, where the former could then just be inf directly. Having both ub and ub_init will be necessary at some point anyway. What would be your opinion on that? Maybe also @dweindl @jvanhoefer ?

A problem with what I just said might be that in sampling, inf bounds make not so very much sense, so having this "unconstrained optimization" in addition may also work.

[FFroehlich]

I believe this should all work fine, no question. However, I was wondering conceptually whether it may be desirable to make the distinction clearer between startpoint and objective bounds, same as it is done in petab (the difference is not supported yet), e.g. by not using the "ConstraintRemover" but instead allowing to pass both ub and ub_init to the problem class, where the former could then just be inf directly. Having both ub and ub_init will be necessary at some point anyway. What would be your opinion on that? Maybe also @dweindl @jvanhoefer ?

A problem with what I just said might be that in sampling, inf bounds make not so very much sense, so having this "unconstrained optimization" in addition may also work.

Oh that actually makes a lot of sense, implementation would also be easier and less hacky. Wasn't aware this was already an (unsupported) petab feature. Will adapt implementation.

[dweindl]

it may be desirable to make the distinction clearer between startpoint and objective bounds, same as it is done in petab (the difference is not supported yet)

Yes, would be great to get that into pypesto. Probably relevant to several people.

allowing to pass both ub and ub_init to the problem class, where the former could then just be inf directly

👍

Besides that, I can't really judge how helpful it will be to have ConstraintRemover on top. Might be convenient for interactive use, but not sure.

[jvanhoefer]

So in general I really like @yannikschaelte's idea of having ub and ub_init separated. Just, isn't this to some degree already possible in PEtab, via specifying ub=inf and specifying an initializationPriorType in the parameter table? (Also isn't at least for PEtab models it already possible to have unconstrained problems?)

[FFroehlich]

So in general I really like @yannikschaelte's idea of having ub and ub_init separated. Just, isn't this to some degree already possible in PEtab, via specifying ub=inf and specifying an initializationPriorType in the parameter table? (Also isn't at least for PEtab models it already possible to have unconstrained problems?)

I think uniform prior with boundaries should be equivalent to specifying ub_init.

[FFroehlich]

Reimplemented using {lb,ub}_init

[yannikschaelte]

Left to do afterwards: Adjust in PEtab import, #522.

[jvanhoefer]

Don't we need to adapt the normalize function to also deal with lb_init and ub_init?

[yannikschaelte]

Don't we need to adapt the normalize function to also deal with lb_init and ub_init?

Yep