Hierarchical Model Fitting

MATLAB code for hierarchical model fitting using iterative importance sampling.

The fitting procedure is demonstrated by fitting RL and Bayesian models to simulated data.

Fitting algorithm

Define model parameters' prior distributions , and repeat until mean likelihood stops increasing:

1. Sample parameter values from
2. Compute likelihood for each sample
3. Derive , by resampling the parameter values with likelihoods as weights
4. Fit prior distributions to the resampled values:

Reinforcement learning model

Parameters: - learning rate

Bayesian inference model

Parameters: - shape of prior distribution

Choice probability (for both models)

Parameters: - inverse temperature

Legend: - expected value, - choice, - trial

Scripts

• demo.m: demonstrates fitting the two models to simulated data, then using one of the models to simulate new data and examine it against the original data
• simulate_data.m: simulate original data from a reinforcement learning agent
• lik_rl.m: reinforcement learning likelihood function (also simulates choices)
• lik_bayes.m: Bayesian inference likelihood function
• mfUtil.m: various functions including the below
  • .randomP - Sample parameter values from prior distributions
  • .computeEstimates - Resample all parameters based on their posterior probability
  • .computeEstimate - Resample one parameter based on its posterior probability
  • .fit_prior - Update the hyperparameters of the prior distribution to reflect the posterior
  • .logsumexp - Compute log(sum(exp(x),dim)) avoiding numerical underflow
  • .randmultinomial - Generate multinomial random numbers
  • .fit2P - organize the results of model fitting for one subject such that they can be fed back into the likelihiid function for simulation