Paring down MeasureBase

[cscherrer]

The current MeasureTheory/MeasureBase partitioning is kind of ad hoc. What we really need is for MeasureBase to contain only

1. Whatever is needed for defining new measures
2. Anything else needed to avoid type piracy

The biggest source of code will be the need to specify base measures. For example, MeasureBase needs to have ProductMeasure, or people won't be able to define anything having a product as a base measure.

Let's identify things not required by the above criteria and discuss moving them to MeasureTheory, to allow MeasureBase to be lighter-weight.

[cscherrer]

Here are some details of the current state, with some ideas for where to go next. Not complete yet, I'll add more details soon

Measures

DensityMeasure

Constructed using ∫. For example, given a measure α and a function f, μ = ∫(f, α) builds a measure μ with density f and base-measure α

PointwiseProductMeasure

Constructed using ⊙. Takes a "prior" measure μ and a Likelihood ℓ, and builds a new measure representing the posterior. This is mathematically equivalent to ∫(ℓ, μ), but in terms of our implementation there's a difference in the base measures. More specifically,

basemeasure(∫(ℓ, μ)) == μ
basemeasure(μ ⊙ ℓ) == basemeasure(μ)

Affine

Represents a pushforward of a measure on ℝⁿ through an affine transformation.

RestrictedMeasure

μ = restrict(f, base) takes f as a predicate for the support. This is like a truncated distribution, but "truncate" has the connatation of renormalization. The implementation is simple:

struct RestrictedMeasure{F,M} <: AbstractMeasure
    f::F
    base::M
end

@inline function logdensity(d::RestrictedMeasure, x)
    d.f(x) || return -Inf
    return 0.0
end

function density(d::RestrictedMeasure, x)
    d.f(x) || return 0.0
    return 1.0
end

Half

Starting with a symmetric univariate measure, this gives a new measure restricted to the half-line. Like RestrictedMeasure, but more efficient.

SuperpositionMeasure

Very much like mixture distributions, but without the reweighting. There are subtleties that come up here for determining the base measure; see the paper for details. Our implementation of this is not yet complete.

SpikeMixture

Superposition of a Dirac with a Lebesgue-dominated measure. Useful for building spike and slab priors, and used in Moritz's "sticky zigzag" sampler.

FactoredBase

It's very common for a log-density with respect to a primitive measure to be sum of three kinds of terms:

1. Data-dependent
2. Parameter-dependent
3. Constant

In addition, the support is often restricted by some predicate.

FactoredBase tries to put all of this base-measure information in one place

struct FactoredBase{R,C,V,B} <: AbstractMeasure
    inbounds::R  # A predicate for membership in the support
    constℓ::C    # A constant 
    varℓ::V      # A nullary function (takes no args) carrying parameter-dependent terms
    base::B      # The base measure
end

@inline function logdensity(d::FactoredBase, x)
    d.inbounds(x) || return -Inf
    d.constℓ + d.varℓ()
end

For example, here's how this can be used for Half:

unhalf(μ::Half) = μ.parent

@inline function basemeasure(μ::Half)
    inbounds(x) = x > 0
    constℓ = logtwo
    varℓ() = 0.0
    base = basemeasure(unhalf(μ))
    FactoredBase(inbounds, constℓ, varℓ, base)
end

I think this is kind of nice, but I'm not yet sure it's the "right" way to set up base measures. We still need to weigh some tradeoffs to get to an approach to advocate as idiomatic.

Kernel

Abstractly, a kernel is a function mapping each value in the domain to a measure. But functions are opaque, and it can be important for us to have more information available statically. So we separate this into two components. In Kernel(f, ops),

• ops maps values (in whatever space) to the measure's "parameters". Exactly what is considered a parameter is measure-dependent. For convenience, ops can also be a named tuple of functions.
• f uses the result of ops to construct the measure. f is typically a UnionAll, but it can also be a generic function.

For example, a Normal approximation to a Poisson could be represented as any of

k1 = λ -> Normal(λ, λ^2)
k2 = kernel(k1)
k3 = kernel(Normal, λ -> (μ=λ, σ=λ^2))
k4 = kernel(Normal) do λ (μ=λ, σ=λ^2) end
k5 = kernel(Normal, (μ = identity, σ = λ -> λ^2))

The information available statically roughly increases at each step (k4 is the exception)

ProductMeasure

This takes a kernel and an array, and returns an abstraction representing the kernel applied to each value. For example,

myprod = ProductMeasure(k, rand(10))

where k is any Kernel above.

TupleProductMeasure

Same, but for tuples

WeightedMeasure

Scale a measure by some positive constant

ParamWeightedMeasure

Similar to a special case of FactoredBase. Not yet in use. Part of exploration to find a good way to represent base measures of parametric measures.

struct ParamWeightedMeasure{L,N,T,B} <: AbstractWeightedMeasure
    ℓ::L
    par::NamedTuple{N,T}
    base::B
end

basemeasure(d::ParamWeightedMeasure) = d.base

logdensity(d::ParamWeightedMeasure, x) = d.ℓ(d.par)

Lebesgue

"Volume" measure. This will probably be refactored to

struct Lebesgue{T} <: PrimitiveMeasure
    dimension :: T
end

Then dimension could be ::Int or ::StaticInt

CountingMeasure

The counting measure of a set is its cardinality. Unless Lebesgue, this will probably never be used directly, but only as a base measure. So we could have

struct CountingMeasure <: PrimitiveMeasure end

Dirac

A point mass, with Dirac(x) ≪ CountingMeasure().

TrivialMeasure

The measure for which every set has measure zero. This is the identity element for superposition.

For any measure μ,

TrivialMeasure() ≪ μ ≪ CountingMeasure()

Measure-Related

Density

AffineTransform

Likelihood

Domains

IntegerRange

RealInterval

Utilities

MapsTo

Iterable

NonIterable

Outdated

Exp

Integral