Implement optuna.terminator using optuna._gp

[y0z]

Motivation

• Change the GP implementation used in optuna.terminator to optuna._gp from BoTorch.

Description of the changes

• Refactoring optuna.terminator and related code for using optuna._gp.
• Remove files that are no longer needed.

[contramundum53]

The result looks good.

import optuna
import numpy as np
from optuna.terminator import RegretBoundEvaluator

study = optuna.study.create_study(sampler=optuna.samplers.TPESampler())

def objective(trial):
    x = trial.suggest_float("x", -3, 3)
    y = trial.suggest_float("y", -3, 3)
    return x ** 2 + y ** 2



study.optimize(objective, n_trials=50)
from optuna.visualization import plot_terminator_improvement

plot_terminator_improvement(study, improvement_evaluator=RegretBoundEvaluator(top_trials_ratio=1.0))

[contramundum53]

LGTM!

[contramundum53]

@nabenabe0928 Could you review this PR?

[contramundum53]

Follow-up tasks:

• Use kernel_params_cache to speed up evaluation
• Set DEFAULT_TOP_TRIALS_RATIO to 1.0
• Set beta to constant

[codecov]

Codecov Report

Attention: 8 lines in your changes are missing coverage. Please review.

Comparison is base (7f70eb2) 89.88% compared to head (5b52d8b) 89.72%.
Report is 15 commits behind head on master.

Files	Patch %	Lines
optuna/terminator/improvement/evaluator.py	89.09%	6 Missing ⚠️
optuna/_gp/prior.py	91.66%	1 Missing ⚠️
optuna/samplers/_gp/sampler.py	75.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #5241      +/-   ##
==========================================
- Coverage   89.88%   89.72%   -0.16%     
==========================================
  Files         213      211       -2     
  Lines       14004    13835     -169     
==========================================
- Hits        12587    12414     -173     
- Misses       1417     1421       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[nabenabe0928]

I confirmed that we get the identical result with my modification as well!

import optuna
import numpy as np
from optuna.terminator import RegretBoundEvaluator

study = optuna.study.create_study(sampler=optuna.samplers.TPESampler(seed=42))

def objective(trial):
    x = trial.suggest_float("x", -3, 3)
    y = trial.suggest_float("y", -3, 3)
    return x ** 2 + y ** 2



study.optimize(objective, n_trials=50)
from optuna.visualization import plot_terminator_improvement

plot_terminator_improvement(study, improvement_evaluator=RegretBoundEvaluator(top_trials_ratio=1.0, seed=42))

[nabenabe0928]

Let's add the version constraints to workflow.yml.

[nabenabe0928]

I was missing this bug until now.

[nabenabe0928]

Thank you for the work, LGTM!

[contramundum53]

LGTM again!