This repo holds the analysis code and data for the cogsci 2021 paper.
How do people ask questions to zero in on a correct answer? Although we can formally define an optimal query to maximize information gain, algorithms for finding this optimal guess may impose large resource costs in space (memory) and time (computation). To understand how people trade off the information gain and the computational difficulty of choosing the ideal query, we turned to a large dataset of 380,000 guesses made during a number-guessing game with Amazon Alexa. We analyzed whether the arithmetic difficulty of following the optimal strategy predicts how far a guess deviates from theoretically optimal query. We find that when memory load is higher, and when more arithmetic operations need to be performed, human guesses deviate more from the most informative query. These results suggest human computational resource constraints limit how people seek out informative questions.
The dataset is from here. The data is under CC BY-SA 4.0.
- cogsci.Rmd: This notebook contains all of the code necessary to replicate the statistics and visualizations used in the paper. Knitting the notebook will run any necessary preprocessing scripts if they haven't already been run.
LaTeX files for generating submitted paper
Data files created and used in analysis.
- games.csv: The original dataset from kaggle.
- *_processed.csv: Files generated by
utils/process_data.R. - guesses_{random,binary}_search.csv: Simulation data generated by
model/scripts.
Generate simulations of guesses using random and binary search.
- binary_search_model.R: Generate guesses using binary search.
- random_search_model.R: Generate guesses using random search.
Helpful scripts used to preprocess and transform data
- get_EIG.R: Core code for calculating EIG of a guess. Used in process_data and model simulation files.
- process_data.R: Filters and transforms the raw data from kaggle.
- read_data.R: Loads data and runs
process_data.Randmodel/files if necessary.