Implement RandomSearch

[ablaom]

This is more-or-less the implementation proposed at JuliaAI/MLJ.jl#37, with addition that tuples of the form (hyperparameter_name, sampler) can be specified in the range vector, where sampler is just something implementing rng.

Here's the updated doc-string:

---

RandomSearch(bounded=Distributions.Uniform,
             positive_unbounded=Distributions.Gamma,
             other=Distributions.Normal,
             rng=Random.GLOBAL_RNG)

Instantiate a random search tuning strategy, for searching over
Cartesian hyperparameter domains, with customizable priors in each
dimenension.

Supported ranges:

• a single one-dimensional range (ParamRange object) r

• a pair of the form (r, d), with r as above and where d is a
  probability vector of the same length as r.values, if r is a
  NominalRange, and is otherwise: (i) any
  Distributions.UnivariateDistribution instance; or (ii) one of
  the subtypes of Distributions.UnivariateDistribution listed in
  the table below, for automatic fitting using Distributions.fit(d, r) (a distribution whose support always lies between r.lower and
  r.upper.)

• any pair of the form (field, s), where field is the (possibly
  nested) name of a field of the model to be tuned, and s an
  arbitrary sampler object for that field. This means only that
  rand(rng, s) is defined and returns valid values for the field.

• any vector of objects of the above form

distribution types	for fitting to ranges of this type
Arcsine, Uniform, Biweight, Cosine, Epanechnikov, SymTriangularDist, Triweight	bounded
Gamma, InverseGaussian, Poisson	positive (bounded or unbounded)
Normal, Logistic, LogNormal, Cauchy, Gumbel, Laplace	any

ParamRange objects are constructed using the range method.

Examples:

range1 = range(model, :hyper1, lower=1, origin=2, unit=1)

range2 = [(range(model, :hyper1, lower=1, upper=10), Arcsine),
           range(model, :hyper2, lower=2, upper=4),
          (range(model, :hyper2, lower=2, upper=4), Normal(0, 3)),
           range(model, :hyper3, values=[:ball, :tree], [0.3, 0.7])]

# uniform sampling of :(atom.λ) from [0, 1] without defining a NumericRange:
struct MySampler end
Base.rand(rng::AbstractRNG, ::MySampler) = rand(rng)
range3 = (:(atom.λ), MySampler(), range1)

Algorithm

Models for evaulation are generated by sampling each range r using
rand(rng, s) where, s = sampler(r, d). See sampler for details. If d
is not specified, then sampling is uniform (with replacement) in the
case of a NominalRange, and is otherwise given by the defaults
specified by the tuning strategy parameters bounded,
positive_unbounded, and other, depending on the NumericRange
type.

See also TunedModel, range, sampler.

[ablaom]

@tlienart A superficial look over would be great, only if you have time.

[tlienart]

it all seems quite sensible to me 👍 I wonder about making this explicit difference between rightunbounded and positiveunbounded shifted by a bit but I'm sure you've already considered all this.

I'm glad to see this anyway 😄

[ablaom]

Many thanks @tlienart .