Implementation of Transitional Markov Chain Monte Carlo (TMCMC) in Julia. This implementation is heavily inspired by the implemntation of TMCMC in OpenCOSSAN.
The TMCMC alorgithm can be used to sample from un-normalised probability density function (i.e. posterior distributions in Bayesian Updating). The TMCMC alorgithm overcomes some of the issues with Metropolis Hastings:
- Can efficiently sample multimodal distributions
- Works well in high dimensions (within reason)
- Computes the evidence
- Proposal distribution selected by algorithm
- Easy to parallelise
Instead of atempting to directly sample from the posterior, TMCMC samples from easy-to-sample "transitional" distributions. Defined by:
where 0 <= Bj <= 1, is iterated in the algorithm starting from Bj = 0 (prior) to Bj = 1 (posterior).
This is not yet a registered julia package. However this package may be installed using the Julia package manager:
julia> ]
pkg> add https://github.com/AnderGray/TransitionalMCMC.jl
Sampling Himmelblau's Function:
using StatsBase, Distributions, PyPlot
using TransitionalMCMC
# Prior Bounds
lb = -5
ub = 5
# Prior Density and sampler
priorDen(x) = pdf(Uniform(lb,ub), x[1,:]) .* pdf(Uniform(lb,ub), x[2,:])
priorRnd(Nsamples) = rand(Uniform(lb,ub), Nsamples, 2)
# Log Likelihood
logLik(x) = -1 .* ((x[1,:].^2 .+ x[2,:] .- 11).^2 .+ (x[1,:] .+ x[2,:].^2 .- 7).^2)
samps, Log_ev = tmcmc(logLik, priorDen, priorRnd, 2000)
plt.scatter(samps[:,1], samps[:,2])
using Distributed, StatsBase, Distributions, PyPlot
addprocs(4; exeflags="--project")
@everywhere begin
using TransitionalMCMC
# Prior Bounds
lb, ub = -5, 5
# Prior Density and sampler
priorDen(x) = pdf(Uniform(lb, ub), x[1,:]) .* pdf(Uniform(lb, ub), x[2,:])
priorRnd(Nsamples) = rand(Uniform(lb, ub), Nsamples, 2)
# Log Likelihood
function logLik(x)
return -1 .* ((x[1,:].^2 .+ x[2,:] .- 11).^2 .+ (x[1,:] .+ x[2,:].^2 .- 7).^2)
end
end
Nsamples = 200
samps, Log_ev = tmcmc(logLik, priorDen, priorRnd, Nsamples, 5, 2)
- Plotting functions
- Storing samples across iterations
- J. Ching, and Y. Chen (2007). Transitional Markov Chain Monte Carlo method for Bayesian model updating, Model class selection, and Model averaging. Journal of Engineering Mechanics, 133(7), 816-832. doi:10.1061/(asce)0733-9399(2007)133:7(816)
- A. Lye, A. Cicirello, and E. Patelli (2021). Sampling methods for solving Bayesian model Updating problems: A tutorial. Mechanical Systems and Signal Processing, 159, 107760. doi:10.1016/j.ymssp.2021.107760
- E. Patelli, M. Broggi, M. D. Angelis, and M. Beer (2014). OpenCossan: An efficient open tool for dealing with epistemic AND Aleatory Uncertainties. Vulnerability, Uncertainty, and Risk. doi:10.1061/9780784413609.258