The domain assertion does not take the parameter transformation into account

[sebffischer]

When evaluating an initial design, generated from a parameter set that uses a trafo, the method eval_batch causes an error because the domain assertion does not take the parameter transformation into account. This can be circumvented by setting objective$check_values = FALSE but I assume the behaviour below is unintented? (This is also a problem for using data transformations in mlr3mbo @sumny )

library(paradox)
library(bbotk)
# define objective function
fun = function(xs) {
  c(y = - (xs[[1]] - 2)^2 - (xs[[2]] + 3)^2 + 10)
}

# set domain
domain = ps(
  x1 = p_dbl(-10, 10),
  x2 = p_dbl(-5, 5)
)

# set codomain
codomain = ps(
  y = p_dbl(tags = "maximize")
)

# create Objective object
objective = ObjectiveRFun$new(
  fun = fun,
  domain = domain,
  codomain = codomain,
  properties = "deterministic"
)

# Define termination criterion
terminator = trm("evals", n_evals = 10)

# create optimization instance
instance = OptimInstanceSingleCrit$new(
  objective = objective,
  terminator = terminator
)

param_set = ps(
  x1 = p_dbl(0, 10, trafo = function(x) x^2),
  x2 = p_dbl(0, 10, trafo = function(x) x^2)
)

design = generate_design_lhs(param_set, 10)$data

instance$eval_batch(design)
#> INFO  [12:21:54.942] [bbotk] Evaluating 10 configuration(s)
#> Error in FUN(X[[i]], ...): Assertion on 'X[[i]]' failed: x2: Element 1 is not <= 5.

^{Created on 2022-01-05 by the reprex package (v2.0.1)}

[sumny]

I would argue that the objective is doing its job as intended: As you specified its domain to be x1 in [-10, 10], x2 in [-5, 5] it tells you that the transformed values are not feasible and complains about it.
If an OptimInstance should have a different search_space than the domain of its objective (e.g., in the case of trafos), the typical way to go would be to pass this ParamSet as the search_space and set check_values = FALSE for the objective (because points may no longer be feasible). Note that the OptimInstance itself still can have check_values = TRUE (default). E.g.:

objective = ObjectiveRFun$new(
  fun = fun,
  domain = domain,
  codomain = codomain,
  properties = "deterministic",
  check_values = FALSE
)

instance = OptimInstanceSingleCrit$new(
  objective = objective,
  terminator = terminator,
  search_space = param_set,
  check_values = TRUE  # TRUE by default but just highlighting this
)

design = generate_design_lhs(instance$search_space, 10L)$data
instance$eval_batch(design)

INFO  [20:33:31.983] [bbotk] Result of batch 1: 
INFO  [20:33:31.986] [bbotk]         x1        x2          y 
INFO  [20:33:31.986] [bbotk]  4.0579007 5.5872638 -1370.1198 
INFO  [20:33:31.986] [bbotk]  2.2821970 6.5024315 -2050.7187 
INFO  [20:33:31.986] [bbotk]  9.5428125 3.7738275 -8219.9013 
INFO  [20:33:31.986] [bbotk]  7.0889246 2.8114950 -2437.2460 
INFO  [20:33:31.986] [bbotk]  3.6864121 1.5835802  -154.6547 
INFO  [20:33:31.986] [bbotk]  5.6997559 9.5832885 -9913.9858 
INFO  [20:33:31.986] [bbotk]  0.1681397 8.8492821 -6605.1702 
INFO  [20:33:31.986] [bbotk]  6.1085515 0.8108775 -1250.4844 
INFO  [20:33:31.986] [bbotk]  8.7620667 7.8375738 -9732.0341 
INFO  [20:33:31.986] [bbotk]  1.8638559 4.2248394  -426.8645 

Via this setup, the actual sampled x1 and x2 are logged as x1 and x2 whereas the actually evaluated transformed values are stored in the x_domain

instance$archive$data[, c("x1", "x2"), with = FALSE]  # sampled x

           x1        x2
 1: 4.0579007 5.5872638
 2: 2.2821970 6.5024315
 3: 9.5428125 3.7738275
 4: 7.0889246 2.8114950
 5: 3.6864121 1.5835802
 6: 5.6997559 9.5832885
 7: 0.1681397 8.8492821
 8: 6.1085515 0.8108775
 9: 8.7620667 7.8375738
10: 1.8638559 4.2248394

instance$archive$data$x_domain  # actually evaluated x based on the trafo

[[1]]$x1
[1] 16.46656

[[1]]$x2
[1] 31.21752

.
.
.

If your use case is something like that you want to frequently switch between ParamSets (i.e., search_spaces within optimization) then I am not sure how we could/should handle this, i.e., switching a search_space during optimization.

I also believe that the separation of domain and search_space like above typically satisfies 99% of use cases in mlr3mbo (excluding the special case that you may want to optimize the instance on a different search_space than the acq_function - this is still WIP but already doable although somewhat hacky).
If you have something else in mind regarding data transformations in mlr3mbo or see any further problems, feel free to open an issue there! I am happy to discuss this.

[sebffischer]

Thanks for the quick response! (There was a copy pasta error in my example but you got it anyway I think) nontheless here the corrected reprex. However, now that I understand the issue better I think I agree with you! Also the solution with setting $check_values = FALSE is good enough for me.

library(paradox)
library(bbotk)
# define objective function
fun = function(xs) {
  c(y = - (xs[[1]] - 2)^2 - (xs[[2]] + 3)^2 + 10)
}

param_set = ps(
  x1 = p_dbl(-10, 10, trafo = function(x) x^2),
  x2 = p_dbl(-5, 5, trafo = function(x) x^2)
)

# set codomain
codomain = ps(
  y = p_dbl(tags = "maximize")
)

# create Objective object
objective = ObjectiveRFun$new(
  fun = fun,
  domain = param_set,
  codomain = codomain,
  properties = "deterministic"
)

# Define termination criterion
terminator = trm("evals", n_evals = 100000)

# create optimization instance
instance = OptimInstanceSingleCrit$new(
  objective = objective,
  terminator = terminator
)


design = generate_design_lhs(param_set, 100)$data

instance$eval_batch(design)
#> INFO  [21:31:44.689] [bbotk] Evaluating 100 configuration(s)
#> Error in FUN(X[[i]], ...): Assertion on 'X[[i]]' failed: x1: Element 1 is not <= 10.

^{Created on 2022-01-05 by the reprex package (v2.0.1)}