BCG #69
BCG #69
Conversation
|
OK this is definitely broken at the moment. Primal values across iterations: |
|
Ok better now: One problem still there is that if tolerance (epsilon) is increased, we end up back at the graph above. But with low tolerance we just oscillate at low values. |
|
Also, I used phi <= epsilon as termination condition, not dual_gap, which is only estimated at steps where we compute a new vertex. |
|
OK this was a bug in the step back step from the simplex descent (when f(y) >= f(x)). Fixed now, converging nicely: |
|
BCG is now working. I will add more sophisticated logging with the number of steps of each type. |
|
looks good - we should also "report" the step in the output, i.e., in the table -> there should be step types already: regular, dual and we might need to add simplex descent |
|
yes I wanted to get this done before merging this PR |
|
perfect - can i run some tests etc? |
|
yes definitely, the tests I ran for now are only partially covering the possible cases |
|
ok will do today |
|
i suppose bcg is also write out trajectory data for evaluation right? |
that's fine -> put you need to use 2phi <= epsilon i suppose as 2phi is the upper bound and phi is used for the step? |
|
just pushed a test example where things repeatedly blew up and added a bit more extra reporting.
|
|
The logging has been updated to include number of non-simplex-GD steps. Things don't "blow up" anymore but the results seem strange (primal value way above the one that seems to appear for vanilla FW) |
|
|
||
| if nforced_fw >= reset_threshold && forced_reset === false | ||
| active_set_cleanup!(active_set, weight_purge_threshold=100*weight_purge_threshold) | ||
| forced_reset = true | ||
| end | ||
|
|
There was a problem hiding this comment.
added cleanup here -> works well in computations now.
There was a problem hiding this comment.
so cleanup works quite well now.
HOWEVER: two more problems
- now the algorithm behaves correctly and only executes SD steps once the active set is "final". does not make progress however. i suppose that this is a line search accuracy issue but could also be because the function value vs. gap spans 1e16 or so and it just might be a numerical issue
- more severely, if you run the "current" example, the algorithm throws tons of warnings. not sure why and where they come from. it is basically line 376+. i suppose this is because the SD should have executed the step and could be the same issue: too small for SD to make progress but then because one of these vertices still have enough weight large enough for afterwards?
There was a problem hiding this comment.
in particular the primal values in this case are not monotonously decreasing -> something is wrong
src/blended_cg.jl
Outdated
| @@ -355,7 +355,7 @@ function lp_separation_oracle( | |||
| kwargs..., | |||
| ) | |||
| # if FW step forced, ignore active set | |||
| if !force_fw_step | |||
| if !force_fw_step && false # TODO: temporarily forced for demonstration | |||
There was a problem hiding this comment.
so here i temporarily force the to run the lmo directly and something weird happens (already beforehand but now more pronounced: the steps are not convergent anymore -> run the example as is to see
|
|
This reverts commit 180ff9f.
So BCG is "working" with quotes. On the example I tried it on in the tests, it finds a solution worse than what vanilla FW finds. I'll re-add logging to see if something comes up