Optimal design of experiments to identify latent behavioral types

This repository contains R code for adaptively constructing and searching an "information landscape" in order to find the particular experimental design that will maximize the information gained as a result of running the experiment. This code accompanies the recent paper:

Optimal design of experiments to identify latent behavioral types
Stefano Balietti, Brennan Klein, and Christoph Riedl,
Experimental Economics 24, pages 772–799 (2021)
arXiv:1807.07024

Parameter-sampled GPUCB-PE

This code samples model parameters and simulates likely datasets that you would observe if experiment participants were parameterized as such. After sampling datasets, each outcome is assigned a likelihood based on its frequency in the sample. Using these likelihoods, the information gain is calculated by taking the Kullback-Leibler Divergence between the likelihoods of the datasets generated under several competing models.

Scripts

1. main.R - start here!
2. simulateDatasets.R - this script houses the Parameter-Sampled G PUCB-PE code
3. models.R - the four models used in our model comparison, including three from El-Gamal & Palfrey (1995)
4. process.R - Gaussian Process script, using functions from GPfit, adapted for optimizing experimental design
5. histories.R - enumerates all possible game histories
6. helper.R - plotting, statistics, and file naming
7. calc_likelihoods.R - used for calculating the likelihoods of each dataset--assuming they have not been sampled
8. matches.R - enumerates possible pairings in the experiment

Citation

If you use these methods and this code in your own research, please cite our paper:

Balietti, S., Klein, B. & Riedl, C. (2021). Optimal design of experiments to identify latent behavioral types,
Experimental Economics 24, pages 772–799

Bibtex:

@article{balietti_optimaldesign_2021,
  title={Optimal Design of Experiments to Identify Latent Behavioral Types},
  author={Balietti, S.; Klein, B. and Riedl, C.},
  journal={Experimental Economics},
  issue={24},
  year={2021},
  pages={772--799}
}

See also:

• El-Gamal, M. A., & Palfrey, T. R. (1996). Economical experiments: Bayesian efficient experimental design. International Journal of Game Theory, 25(4), 495-517. doi: 10.1007/BF01803953.
  • motivating work from which many of these ideas derive