Independent ask and tell

[Dronablo]

Steps to reproduce

1. I have a function to minimize, which is extremely slow.
2. Due to the fact, that function has several steps inside, I could detect bad outcome early and return some big constant, say 10e5 (possible optima is 0).
3. Also, for some parameters combination it's impossible to get any result, so I also return 10e5.
4. To optimize such a function with nevergrad I need an ability to call ask and tell completely independent. Is it possible?

Relevant Code

What do I want could be coded like:

for param, value in somehow_before_obtained_results:
  optimizer.tell(param, value)

for _ in range(optimizer.budget):
  x = optimizer.ask()
  value = square(*x.args, **x.kwargs)
  if value<10e5:
      optimizer.tell(x, value)
recommendation = optimizer.provide_recommendation()

[jrapin]

Something like the following should work:

for param, value in somehow_before_obtained_results:
  candidate = optimizer.create_candidate.from_call(*params)
  # may fail for some optimizers who do not support tell without ask (but most are OK)
  optimizer.tell(candidate, value)  

for _ in range(optimizer.budget):
  x = optimizer.ask()
  value = square(*x.args, **x.kwargs)
  optimizer.tell(x, value)
recommendation = optimizer.provide_recommendation()

I suppose your question mostly concerned the if condition. I removed it because while some algorithms could deal with it, all population based algorithm need to have a tell for each ask (otherwise the population does not get updated, and the algorithm blocks at some point). However if you detect the value early, why dont you just send a large value instead of skipping the tell? This is also more informative for the optimizer.

[Dronablo]

Well, I actually send =).
I tried it with BayesianOptimization library, and it leads to stuck the process: optimizer starts to return params only from the edges (min/max for every dimension).
I will try it with nevergrad.

Thanks a lot!

[jrapin]

All BO based algorithms in nevergrad use BayesianOptimization under the hood, so you will probably have the same problem if you use them, but most other optimizers should be fine. TwoPointsDE and OnePlusOne are usually safe bets.

[Dronablo]

So, I've updated nevergrad to 0.4.0.
Suggested way of creating candidates optimizer.create_candidate.from_call() was removed from code, but still present in tell method description.
As I could find from changelog and repo code I should use something like instrum.spawn_child().set_standardized_data() now. Could I ask someone to provide an example?

[jrapin]

Hi,
Sorry this is proving difficult, I tried to fix and improve the documentation regarding inoculation in #595.
I merged it to ease the visualization in the documentation, it should be live:

• https://facebookresearch.github.io/nevergrad/optimizers_ref.html#nevergrad.optimizers.base.Optimizer.tell
• https://facebookresearch.github.io/nevergrad/optimization.html#telling-non-asked-points-or-suggesting-points

Does that answer your question?
In any case, thank you for flagging the erroneous documentation, and let us know if you have ideas of how to make it clearer ;)

[Dronablo]

Thanks a lot, now it's clear.

As I could guess from changelog, there is some change in points storage mechanic in recent times also: #571. If this is right, there is one more uncertain part.

• Let's assume I've done complete nevergrad run and got some output - let's call it best_y and best_args.
• After some time I've got some more data and want to make new nevergrad run.

Could you please say am I right or not for the following statements:

• Doing one hand-made tell call from last (best_args, best_y) is better than none.
• Doing several (2-10-50-100?) hand-made tell calls from previous runs is better than one.
• Doing more than 1000 calls is useless.
• Doing hand-made calls for bad outcomes (bad parameter combinations, early stops) is somewhat useless too.
  ?

[jrapin]

Doing one hand-made tell call from last (best_args, best_y) is better than none.

Yes, but not all optimizers support tells that where not asked. (I'll provide another option at the end)

Doing several (2-10-50-100?) hand-made tell calls from previous runs is better than one.

Somehow: some optimizers (like most standard variants of OnePlusOne) only use the one best point so far, so more than 1 may not be necessary, but wont hurt. Other optimizers use a population of points (DE variants, TBPSA, CMA), usually in our implementations this is around 40 points (though that may come to change, 40 is basically a good enough default usually, but hopefully we can find something more adaptative eventually). Those optimizers will keep the 40 bests or so if you tell them more.

Doing more than 1000 calls is useless.

Indeed, we wont keep track of them. Some non-standard variants of OnePlusOne do crossovers with points in the archive, which size I recently decreased as you mentioned, but that's nearly the only one that could maybe make some use of it, and it's not sure it would be a good use.

Doing hand-made calls for bad outcomes (bad parameter combinations, early stops) is somewhat useless too.?

Indeed, they are dropped for most optimizers. However now that you mention it, BO optimizers may be able to make use of all the information you provide since they have some underlying model. In this case you the more tell you do the better I would assume. I've never been a fan of BO though.

Another option to consider: setup your parametrization to have the initialization you want. With the new parametrization you can set the value manually, and it will be used as starting/reference point. You should also update the mutation sigma (param.set_mutation(sigma=x)), which is basically equivalent to a step size. It is initialized to 1 by default.

Does that clarify it for you? If you see specific areas of the documentation I could clarify don't hesitate to point it out. I don't know how to integrate all these information in a clear way :s

[Dronablo]

A lot of useful info, thank you!

setup your parametrization to have the initialization you want. With the new parametrization you can set the value manually

I'm doing it in the following manner:

instr_dict = {
    'param1':
    ng.p.Scalar(init=best_args['param1'],
                lower=param_bounds['param1'][0],
                upper=param_bounds['param1'][1]),
    'param2':
    ng.p.Log(init=best_args['param2'],
             lower=param_bounds['param2'][0],
             upper=param_bounds['param2'][1]),
    'param3':
    ng.p.TransitionChoice(
        np.arange(param_bounds['param3'][0], param_bounds['param3'][1]))
}

instrum = ng.p.Instrumentation(**instr_dict)

for args, val in previous_outputs:
    cand = instrum.spawn_child(new_value=(tuple(), args))
    optimizer.tell(cand, val)

Do you advice me to do

instrum.value = (tuple(), best_args)

After? It throws RuntimeError: Cannot modify frozen Parameter. Does it really neded if I set init arguments in instrum?

I don't know how to integrate all these information in a clear way

I found your last answer #263 (comment) is a candidate to be completely transferred to https://facebookresearch.github.io/nevergrad/optimization.html#telling-non-asked-points-or-suggesting-points

[jrapin]

After? It throws RuntimeError: Cannot modify frozen Parameter. Does it really needed if I set init arguments in instrum?

Parameters are "frozen" as soon as they interact with the optimizer, to avoid side effects. Setting the value of the parametrization does not raise the exception if you do it before instantiating the optimizer. I'll probably need to explain this "frozen" part in the doc too :D

If you have provided init values then you are all fine though. Note that by default, when you set lower and upper bounds, sigma (= the step size) is set to 1/6th of the full range. You may want to update it if you have a better prior.