Simulation code for the computational study by Diekmann and Cheng (2023) published in eLife:
A model of hippocampal replay driven by experience and environmental structure facilitates spatial learning
https://doi.org/10.7554/eLife.82301
Install Requirements
-
First, download and install CoBeL-RL:
https://github.com/sencheng/CoBeL-RL -
Install python packages required for plotting:
pip3 install -r requirements.txt
Demo Guide
Demos can be found in the 'demo' directory. Run the SFMA demo to ensure that you installed CoBeL-RL properly.
-
Navigate to
demo/. -
Run the script
sfma_demo.pyto run the SFMA demo. -
Run the script
deep_sfma_demo.pyto run the Deep SFMA demo.
Note: The Deep SFMA demo requires Tensorflow 2 to be installed.
Simulations Guide
Simulations can be found in the 'simulations' directory. The following sections will guide you through how to run specific simulations.
Learning Simulation
-
Navigate to
simulations/learning/. -
Run the script
make_environments.pyto generate the gridworld environments. -
Run the script
simulation_learning.pyto run the learning simulation. -
Run the script
analyze_learning.pyto analyze and plot the learning simulation data.
Optimality Simulation
This simulation is identical to the learning simulation but only runs simulations for the SFMA agent and records the optimality of updates.
-
Navigate to
simulations/optimality/. -
Run the script
make_environments.pyto generate the gridworld environments. -
Run the script
simulation_optimality.pyto run the optimality simulation. -
Run the script
analyze_optimality.pyto analyze and plot the optimality simulation data.
Sequences Simulation
This simulation is identical to the learning simulation but only runs simulations for the SFMA agent and records the replays.
-
Navigate to
simulations/sequences/. -
Run the script
simulation_sequences.pyto run the sequences simulation. -
Run the script
analyze_sequences.pyto analyze and plot the sequences simulation data. -
Run the script
plot_replays.pyto plot the replays.
Random Walk Simulation
Simulation of the experiments by Stella et al. (2019).
-
Navigate to
simulations/random_walk/. -
Run the script
precompute_occupancies.pyto generate occupancy data. -
Run the script
simulation_random_walk.pyto run the random walk simulation. -
Run the script
simulation_distribution.pyto generate data for the replay starting point and displacement distribution analysis. -
Run the script
analyze_random_walk.pyto analyze and plot the random walk simulation data. -
Run the script
analyze_distribution.pyto analyze and plot the replay starting points and displacement distribution.
Shortcuts Simulation
Simulation of the experiments by Gupta et al. (2010).
-
Navigate to
simulations/shortcuts/. -
Run the script
simulation_shortcuts_static.pyto run the shortcuts simulation. -
Run the script
simulation_shortcuts_learning.pyto run the shortcuts learning simulation. -
Run the script
simulation_shortcuts_strengths.pyto record experience strengths. -
Run the script
analyze_shortcuts_static.pyto analyze the shortcuts simulation data. -
Run the script
analyze_shortcuts_learning.pyto analyze and plot the shortcuts learning simulation data. -
Run the script
plot_shortcuts_static.pyto plot the analyzed shortcuts learning simulation data. -
Run the script
plot_shortcuts_static_replays.pyto plot detected shortcut replays. -
Run the script
plot_shortcuts_strengths.pyto plot the experience strengths.
Dynamic Simulation
Simulation of the experiments by Widloski and Foster (2022).
-
Navigate to
simulations/dynamic/. -
Run the script
simulation_dynamic.pyrun the dynamic simulation. -
Run the script
analyze_dynamic.pyto analyze and plot the dynamic simulation data. -
Run the script
compare_dynamic.pyto compare results for different experience similarity metrics.
Preplay Simulation
Simulation of the experiments by Ólfasdóttir et al. (2015).
-
Navigate to
simulations/preplay/. -
Run the script
simulation_preplay.pyrun the preplay simulation. -
Run the script
analyze_preplay.pyto analyze the preplay simulation data. -
Run the script
analyze_arms.pyto plot the analyzed preplay simulation data. -
Run the script
simulation_learning.pyrun the preplay learning simulation. -
Run the script
analyze_learning.pyto analyze and plot the preplay learning simulation data.
Reward Simulation
Simulation for demonstrating the effect of reward on replay statistics.
-
Navigate to
simulations/reward/. -
Run the script
simulation_reward.pyrun the reward simulation. -
Run the script
analyze_reward.pyto analyze and plot the reward simulation data.
Aversive Simulation
Simulation of the experiments by Wu et al. (2017).
-
Navigate to
simulations/aversive/. -
Run the script
simulation_aversive.pyrun the aversive simulation. -
Run the script
analyze_aversive.pyto analyze and plot the aversive simulation data.
Reward Change Simulations
Simulation of the experiments by Ambrose et al. (2016).
-
Navigate to
simulations/reward_change/. -
Run the script
simulation_effect_of_reward.pyrun the effect of reward simulation. -
Run the script
analyze_effect_of_reward.pyto analyze and plot the effect of reward simulation data.
Goal Change Simulation
Simulations for the learning performance following reward changes in the different replay modes.
-
Navigate to
simulations/goal_change/. -
Run the script
simulation_changingGoal.pyrun the goal change simulation. -
Run the script
analyze_changingGoal.pyto analyze and plot the goal change simulation data.
Detailed Replay Simulation
Simulation recording replay step-by-step (for demonstration purposes).
-
Navigate to
simulations/detailed_replay/. -
Run the script
simulation_detailed_replay.pyrun the detailed replay simulation. -
Run the script
plot_detailed_replay.pyto plot step-by-step replay.
Modes Simulation
Simulation demonstrating the possible replay modes.
-
Navigate to
simulations/modes/. -
Run the script
simulation_modes.pyrun the modes simulation. -
Run the script
plot_modes.pyto plot examples for experience similarity and generated replays.
Non-local Simulations
Simulation demonstrating non-local online (awake) replay.
-
Navigate to
simulations/nonlocal/. -
Run the script
simulation_gupta.pyrun the non-local simulation in the figure-8 maze. -
Run the script
simulation_open_field.pyrun the non-local simulation in an open field. -
Run the script
analyze_gupta.pyto analyze and plot the replays in the figure-8 maze. -
Run the script
analyze_open_field.pyto analyze and plot the replays in an open field.
PMA Gain Simulation
Simulation demonstrating how adjusting the gain computation in the model by Mattar and Daw (2018) affects replay.
-
Navigate to
simulations/pma_gain/. -
Run the script
simulation_distribution.pyrun the PMA Gain simulation. -
Run the script
analyze_replay.pyto analyze and plot the replays.