Provide default/good hyperparameters to start search

[JohnGiorgi]

Is there any way to provide a set of default hyperparameters to start the search?

The motivation would be that you already have a sense of "good" hyperparameters, so beginning the search with them may be far more efficient, especially when using a sampler.

[himkt]

Thank you @JohnGiorgi for raising the interesting issue.

You would like to set a fixed parameter set like the following?:

import optuna


def objective(trial):
    x = trial.suggest_float("x", 0, 10)
    return x ** 2


study = optuna.create_study()
study.enqueue_trial({"x": 5})  # set a parameter x=5 for the first trial
study.enqueue_trial({"x": 0})  # then set x=0 for the next trial
study.optimize(objective, n_trials=2)

assert study.trials[0].params == {"x": 5}  # x=5 in the first trial
assert study.trials[1].params == {"x": 0}  # x=0 in the second trial

https://optuna.readthedocs.io/en/stable/reference/generated/optuna.study.Study.html#optuna.study.Study.enqueue_trial

If my understanding is correct, the answer is (partially) yes. allennlp-optuna doesn't provide a way to set predefined parameter combinations but we can load an existing study by --skip-if-exists option. So I think we can realize that by the following two steps:

1. create study and set manual hyperparameters in Python

> python
>> import optuna
>> study = optuna.craete_study(storage="sqlite:///sample.db", study_name="some_name")
>> study.enqueue_trial({"x1": your_awesome_parameter, "x2": your_awesome_parameter})
>> exit()

2. use the study in allennlp-optuna

allennlp tune ... --study some_name --storage sqlite:///sample.db --skip-if-exists

[JohnGiorgi]

@himkt That's great! This is exactly what I was looking for. I will try this out :)

[himkt]

@JohnGiorgi Let me close this issue. Please feel free to reopen if you need! 👋