GPopt (R)

This is the R version of Python's Techtonique/GPopt, a package for 'Bayesian' optimization of black-box functions (and machine learning hyperparameter tuning) using Gaussian Process Regression and other surrogate models.

It's ported the same way as nnetsauce for R was: with uv to create an isolated Python virtual environment containing the Python GPopt package, and reticulate to call into it from R. Every function in this R package is a thin wrapper that returns the underlying Python object; the general rule is: object accesses with .'s in Python are replaced by $'s in R.

See this post for the technique: Finally figured out a way to port python packages to R using uv and reticulate.

Install

1. Create the Python virtual environment with uv

# pip install uv # if necessary
uv venv venv
source venv/bin/activate      # on Windows: venv\Scripts\activate
uv pip install pip GPopt

Keep track of where venv/ lives -- you'll pass its path as venv_path to every function in this package.

2. Install the R package

install.packages("remotes")
remotes::install_github("Techtonique/GPopt_r") # or wherever this package is hosted

reticulate will be installed automatically as a dependency.

If you encounter errors on Windows, consider using the Windows Subsystem for Linux.

Examples

Minimizing the Branin function

library(GPopt)

branin <- function(x) {
  x1 <- x[1]; x2 <- x[2]
  term1 <- (x2 - (5.1 * x1^2) / (4 * pi^2) + (5 * x1) / pi - 6)^2
  term2 <- 10 * (1 - 1 / (8 * pi)) * cos(x1)
  term1 + term2 + 10
}

opt <- GPOpt(
  lower_bound = c(-5, 0),
  upper_bound = c(10, 15),
  objective_func = branin,
  n_init = 10,
  n_iter = 10,
  venv_path = "./venv"
)

opt$optimize(verbose = 1L)
print(opt$x_min)  # current best parameters
print(opt$y_min)  # current best objective value

# resume with more iterations
opt$optimize(verbose = 1L, n_more_iter = 10L)

Tuning a scikit-learn model's hyperparameters

library(GPopt)

sklearn <- get_sklearn(venv_path = "./venv")
RandomForestClassifier <- sklearn$ensemble$RandomForestClassifier

X <- as.matrix(iris[, 1:4])
y <- as.integer(iris$Species) - 1L

mlopt <- MLOptimizer(scoring = "accuracy", cv = 5, venv_path = "./venv")

param_config <- list(
  n_estimators = list(bounds = c(10, 300), dtype = "int"),
  max_depth    = list(bounds = c(1, 20),   dtype = "int")
)

mlopt$optimize(
  X_train = X, y_train = y,
  estimator_class = RandomForestClassifier(),
  param_config = param_config,
  verbose = 1L
)

print(mlopt$get_best_parameters())
print(mlopt$get_best_score())

Bayesian optimization with early stopping

library(GPopt)

opt <- BOstopping(
  f = branin,
  bounds = rbind(c(-5, 10), c(0, 15)),
  venv_path = "./venv"
)
result <- opt$optimize(n_iter = 100L)

Using a custom (conformalized) surrogate model

library(GPopt)

sklearn <- get_sklearn(venv_path = "./venv")
base_model <- sklearn$ensemble$ExtraTreesRegressor()
surrogate <- GenericSurrogate(base_model, conformal = TRUE, venv_path = "./venv")

opt <- GPOpt(
  lower_bound = c(-5, 0),
  upper_bound = c(10, 15),
  objective_func = branin,
  surrogate_obj = surrogate,
  venv_path = "./venv"
)
opt$optimize(verbose = 1L)

Functions in this package

• GPOpt() -- main Bayesian optimizer (wraps GPopt.GPOpt)
• MLOptimizer() -- cross-validated hyperparameter tuning for scikit-learn estimators (wraps GPopt.MLOptimizer)
• GenericSurrogate() -- wrap any scikit-learn-compatible regressor as a surrogate (wraps GPopt.GenericSurrogate)
• GeneralizationOpt() -- diagnostics for the generalization gap of hyperparameters (wraps GPopt.GeneralizationOpt)
• BOstopping() -- Bayesian optimization with early stopping (wraps GPopt.BOstopping)
• get_GPopt(), get_numpy(), get_sklearn() -- access the raw Python modules for anything not (yet) covered above

Notes

• Every function takes a venv_path argument pointing at the uv-created virtual environment; this must be the same environment across a given script/session for object identity between calls to work correctly.
• R functions passed as objective_func (or f) are called from Python with a 1-D numpy array converted back to an R numeric vector -- so index with x[1], x[2], ... as usual in R, not x[0], x[1], ...
• Since this package hands you the underlying Python object, the full Python GPopt API is available -- consult the Python package's documentation and blog posts for anything not covered in this README.

License

BSD-3-Clause-Clear, matching the original Python GPopt package and nnetsauce_r.