Nested parameter definitions

[vollmersj]

Nested parameter sets are currently not supported we could borrow from JuMP
ps = makeParamSet(

makeDiscreteParam("kernel", values=c("polydot", "rbfdot")),

makeNumericParam("C", lower=-15, upper=15, trafo=function(x) 2^x),

makeNumericParam("sigma", lower=-15, upper=15, trafo=function(x) 2^x,

requires = quote(kernel == "rbfdot")),

makeIntegerParam("degree", lower = 1, upper = 5,

requires = quote(kernel == "polydot")))

[ablaom]

This is how I would handle nested parameters:

Nested lists of pairs, constructed using a new Pairs type, are used
to represent both fully expanded "values" of some (possibly composite)
model, as well as the iterators generating values to be resampled.

So, suppose we have a composite model with declaration

mutatable struct SomeComposite <: Supervised
    outer1
    outer2
    model::SomeModel
end

where outer1 and outer2 are numerical and SomeModel has declaration

mutable struct SomeModel <: Supervised
    inner1
    inner2
end

Then for some composite::SomeComposite the call get_params(composite) returns

Pairs(:outer1 => 2.3, :outer2 => 42, :model => Pairs(:inner1 => 1, :inner2 => -1))

There will be a corresponding set_params! method which mutates a given model according to the nested sequence passed (which need not be "fully populated" - we can just change some of the parameters of our model).

To specify ranges of parameter values we specify iterators (either deterministic or
stochastic) instead of values. Supposing we only want to tune some parameters, we might set

grid = Pairs(:outer2 => iter_out2, :model => Pairs(:inner1 => iter_in1))

In place of iterators we could use the existing
makeNumericalParam(), etc, constructs, from which iterators of the right kind could then generated.

To carry out tuning, all we need are two other methods I can write
out: (i) one to flatten out values (without the symbolic keys) in a
nested sequence of pairs (returns a tuple of values); and (ii) one to
insert flat values into (a copy of) some compatible nested sequence (which can then be used to mutate a model object with set_params!)

The reason we need the Pairs type instead of just using nested
tuples is that we need distinguish bewteen tuples describing our nested
structure from tuples that are actually parameters of some kind.

(I'm not stuck on the name Pairs. It's just that a => b has type Pair in Julia.)

[ablaom]

Closing as no objections raised and now implemented.