Comparison to TransformVariables and Bijectors?

[cscherrer]

Hi, from the README this looks very similar to TransformVariables, and to some special cases of Bijectors. How is this package different from those? It would be helpful to have some details on this in the README, to help users know which of the three packages is the best fit for their use case.

[willtebbutt]

I confess that I had forgotten that TransformVariables exists, but it does seem to fill a slightly different niche from this package at present, but it's not impossible that there's a way to unify the two packages.

AFAICT the primary conceptual differences are that ParameterHandling.jl is interested in completely arbitrary data structures, whereas TransformVariables.jl is interested specifically in the bijections utilised when working with random variables, and ParameterHandling breaks down into two steps what TransformVariables does in one.

TransformVariables

IIUC, TransformVariables.jl exposes (in addition to a number of transforms, and operations on those transforms) one function for constructing transformations which map from vectors of reals to other data structures, whose values respect certain constraints:

# My understanding based on these docs: https://tamaspapp.eu/TransformVariables.jl/stable/#The-as-constructor-and-aggregations
t = as(T, data...) # "constructor"
t = as(data....) # used for "aggregations"

t is then a callable which maps from dimension(t) to a structured + constrained representation of the data. For example,

as(Real, 0.0, ∞)

maps the real line to the positive reals, and

t = as((μ = asℝ, σ = asℝ₊, τ = asℝ₊, θs = as(Array, 8)))

maps from 11-dimensional Euclidean space to a NamedTuple.

My suspicion is that TransformVariables is in principle able to operate on fairly arbitrary data structures -- my impression is that the current focus is numbers, arrays or numbers, and named tuples.

ParameterHandling

ParameterHandling.jl exposes a two-function API, rather than one:

x_flat, unflatten = flatten(x)
ParameterHandling.value(x)

flatten maps an arbitrary data structure to a vector and a closure (known by convention as unflatten) satisfying:

x_flat isa Vector{<:Real}
unflatten(x_flat) == x

flatten is purely a "rearranging" operation -- it takes a data struture containing numbers and constructs a vector containing those numbers in a deterministic manner. unflatten is the inverse of this "rearranging" operation.

Separately, ParameterHandling.jl defines a collection of things which subtype AbstractParameter. For example

x = positive(5.0)
x isa ParameterHandling.Positive
ParameterHandling.value(x) == 5.0

x represents a positive number. Upon calling positive, 5.0 is mapped to the real line via Bijectors.Log (or something like that), and placed inside a Positive <: AbstractParameter. When ParameterHandling.value is called upon a Positive, Bijectors.exp is called on the wrapped value, and its output returned.

ParameterHandling.value is defined to either be the identity for things like Reals and Array{<:Real}s, and is defined recursively to things like Tuple, NamedTuple, and Dict.

One could implement the example from TransformVariables in ParameterHandling as follows:

# TransformVariables version.
t = as((μ = asℝ, σ = asℝ₊, τ = asℝ₊, θs = as(Array, 8)))

# ParameterHandling vesion.
x_flat, unflatten = flatten((
    μ=5.0,
    σ=positive(2.0),
    τ=positive(1.1),
    θs=randn(8),
))

# The same transformation in either framework.
x_tv = t(x_flat)
x_ph = ParameterHandling.value(unflatten(x_flat))

all(map(isapprox, x_tv, x_ph)) # return true

Notable Differences

These APIs are really quite similar, but there are a few differences. The TransformVariables interface allows one to specify the transform without providing an initial value, whereas ParameterHandling builds the transform from the initial value. For the simple examples above, this distinction isn't particularly important. It's possible that the ParameterHandling interface is a little more convenient in that it provides the initial value in the transformed space, whereas I think you'd have to separately write x_flat = TransformVariables.inverse(t)(x) to get it using TransformVariables.

The ParameterHandling approach has the minor advantage that things like Vector{Float64}s that don't have any particular constraints don't need to have any additional stuff attached to them (no call to as) -- you can just put them in your data structure as-is.

I think the differences start to really show when you consider doing something more compicated like

x = deferred(Normal, 5.0, positive(1.0))
x_flat, unflatten = flatten(x)
ParameterHandling.value(x)

The deferred function constructs a Deferred object which is just another AbstractParameter which eats a callable and its arguments, and whose value is produced by first calling ParameterHandling.value on its arguments, then calling the callable on said arguments. The point of this is to allow the user to work with arbitrary data structures that know nothing about ParameterHandling -- Normal knows nothing about Positive parameters.

Presumably TransformVariables could handle stuff like this with some modification / extension, but I'm not completely sure how you would make sense of things like logdetjacobian etc in that context. Probably there's a reasonable definition.

Conversely, ParameterHandling doesn't have anything to say about logdetjacobian, but it could certainly be extended to do so in cases where it makes sense.

The reason that we went down the route of requiring that you provide us with an actual piece of data to apply flatten and value to in ParameterHandling was because it avoids having to add functionality to specify "this thing will build a transformation which maps from a vector of reals to this type with these dimensions etc". If you actually have a bit of data, you know precisely what structure the transformation should map to.

@cscherrer does this make sense?

[cscherrer]

Thanks @willtebbutt , this is really helpful. It seem like this approach could make it more flexible about types, so you could put easily use StaticArrays, for example. Is that right?

deferred looks really useful! Very nice way of writing things :)

For calls like positive(2.0), is the value just a way of getting the type, or does it serve another purpose?

Given the similarities to TransformVariables, do you think there is a nice way to have them under a common interface?

The reason that we went down the route of requiring that you provide us with an actual piece of data to apply flatten and value to in ParameterHandling was because it avoids having to add functionality to specify "this thing will build a transformation which maps from a vector of reals to this type with these dimensions etc". If you actually have a bit of data, you know precisely what structure the transformation should map to.

This is great. We're working through some similar issues in MeasureTheory, and I've started working more and more in terms of testvalues, with basically the same motivation.

Thanks for the detailed description!

[willtebbutt]

It seem like this approach could make it more flexible about types, so you could put easily use StaticArrays, for example. Is that right?

Exactly. Indeed, some SparseMatrixCSC functionality was recently added.

For calls like positive(2.0), is the value just a way of getting the type, or does it serve another purpose?

Well in the particular case of positive it really is about the type, but more generally info about the data itself is needed to know how to implement flatten and value.

Given the similarities to TransformVariables, do you think there is a nice way to have them under a common interface?

Probably. You could certainly implement TransformVariables in terms of ParameterHandling -- maybe the converse is true also? I wonder whether @tpapp has any thoughts?