v0.44.0

Turing v0.44.0

Diff since v0.43.7

Breaking changes

Variational inference interface

The VI interface in Turing has been modified to make it more interoperable with the rest of Turing.

• The arguments to vi(...) are slightly different: instead of specifying a q_init argument (the initial variational approximation), you now directly pass a function that constructs this for you. For example, instead of

  q_init = q_meanfield_gaussian(model)
  vi(model, q_init, n_iters)

  you would now do

  vi(model, q_meanfield_gaussian, n_iters)

• The return value of vi is now a VIResult struct (please see the documentation for information), which bundles the previous return values together in a more cohesive way.
  Most importantly, you can now call rand([rng,] result::VIResult) to obtain new samples from the variational approximation.
  This returns a VarNamedTuple of raw values, which can be used directly in all other Turing interfaces without any further wrangling.
  (In contrast, the previous return value of rand(q) would yield a vector of transformed parameters.)

Internally, the VI interface has been reworked to directly use DynamicPPL.LogDensityFunction instead of relying on a transformed distribution from Bijectors.jl.

Gibbs sampler interface

This section is only relevant if you are writing a sampler that is intended to be directly used as a component sampler in Turing's Gibbs sampler.
(If Gibbs calls your sampler via Turing's externalsampler interface, this section does not apply to you.)

Turing's Gibbs sampler has been reworked in this release to fix a number of correctness and performance issues.
The main change is that the Gibbs state carries a VarNamedTuple of raw values, instead of a VarInfo of vectorised (transformed) parameters.
This fixes correctness issues that arise with value-dependent transformations, and also leads to much reduced overhead when sampling with Gibbs: see #2803 for some representative benchmarks.

In Turing v0.43, you would have to define two methods

• Turing.Inference.get_varinfo(::MyState) -> returns a VarInfo of values from your sampler's state

• Turing.Inference.setparams_varinfo!!(::DynamicPPL.Model, ::MySampler, ::MyState, params::AbstractVarInfo) -> uses a VarInfo of values to update your sampler's state

The corresponding methods in Turing v0.44 are

• Turing.Inference.gibbs_get_raw_values(::MyState) -> returns a VarNamedTuple of raw values from your sampler's state. Note that these values should not be transformed or wrapped in any way: if x is 3 in the model, then we should have that gibbs_get_raw_values(state)[@varname(x)] == 3.

• Turing.Inference.gibbs_update_state!!(::MySampler, ::MyState, ::DynamicPPL.Model, global_vals::VarNamedTuple) -> uses the values in global_vals to update your sampler's state and return a new state.
  Note that the values in global_vals are raw values.
  Also, note that the model argument passed in here will be 'conditioned' on the new values inside global_vals.
  That means that if any part of your state relies on caching the model being evaluated, you have to update those parts of your state to reflect the new model as well.

As can be seen, the interface is almost entirely the same except that we use VarNamedTuples of raw values instead of VarInfos of potentially transformed parameters.

Please see the docstrings in the Turing.jl API page for more information.

Other changes

Performance

Previously many of Turing's samplers needed to carry around a VarInfo so that they could communicate with Gibbs.
This release frees them up from having to do so, and in particular allows for usage of DynamicPPL.OnlyAccsVarInfo, which is much cheaper as it avoids unnecessary computations.

As a result, samplers such as MH and ESS are faster in this release, sometimes by up to 5x.

Fixed transforms

The MCMC sampling (sample), optimisation (mode_estimate / maximum_likelihood / maximum_a_posteriori, and VI (vi) entry points now accept an extra fix_transforms keyword argument, which specifies that all transforms in the model should be determined once at the start of inference and then fixed to those values for the rest of inference.
(In contrast, the default behaviour is to rederive transforms each time the model is run.)

Note that not all MCMC samplers currently support fixed transforms.
In particular, HMC, NUTS, and external samplers currently do support it, and Gibbs will pass the flag through to its component samplers, but MH's LinkedRW option will currently ignore the fix_transforms argument.
For some samplers such as ESS and particle MCMC, fixed transforms do not affect the sampling process at all (in such cases the keyword argument is accepted but ignored).

The reason why Turing rederives transforms is to ensure correctness in cases where the transform depends on the value of another random variable.
For example, if a is a parameter, then b ~ Uniform(-a, a) has a transform that depends on the value of a.
Since a is not static, this in turn means that the transform associated with b is not static.

If you know that this consideration is not relevant for you, it can sometimes lead to performance benefits if the transforms are expensive to compute.
However, in many cases the benefits are negligible and you should always benchmark this on a case-by-case basis.
Please see https://turinglang.org/DynamicPPL.jl/stable/fixed_transforms/ for further details about this.

Merged pull requests:

• (breaking) v0.44 (#2790) (@penelopeysm)
• Rework of Gibbs sampler interface (#2803) (@penelopeysm)
• Remove unnecessary hacks in Gibbs interface (#2811) (@penelopeysm)

Closed issues:

• All VarInfos are informative, but some VarInfos are more informative than others (#2642)
• Can we get rid of differentiating through GibbsContext? (#2761)
• ESS can use LogDensityFunction throughout (#2762)
• Gibbs interface (#2764)
• VI interface is too low-level (#2783)
• dynamic transform correctness issue (#2801)
• enable fixed transforms for MCMC & optimisation (#2806)