Code for a model of the striosome→dopamine→matrix circuit where dopamine encodes “policy-information-gain” (policy-IG), i.e. the reduction in policy entropy from max possible entropy. All code is written in python 3.13. Below, we map all figures with code to files. All other figures are cartoons drawn in Adobe Illustrator. We do not use any external data for the project.
Fig. 2d-g: Associative_learning_correlations/associative_learning_RPE_pIG_correlation.py
Fig. 3b,d,f: Associative_learning_correlations/associative_learning_RPE_pIG_large_cost_and_novelty.py
Fig. 3g: Associative_learning_correlations/theoretical_3D_curve.py
Fig. 4b,d: Ramping_and_state_value/basic_ramping_and_teleport.py
Fig. 4e,f: Ramping_and_state_value/probabilistic_task_ramping.py
Fig. 5a: Action_initiation/policy_shift_action_initiation.py
Fig. 5d: Action_initiation/hierarchical_melody_note_primative_simulation.py
Fig. 6a-c: Decision_making_biases/risk_impulsivity_and_conflict.py
Fig. 6d: Decision_making_biases/risk_impulsivity_and_conflict.py/Ushapeperformance.py
Fig. 7f,g: Disorders/coupled_infoIG_RL_sim.py
Fig. 8b: Circuit_design/ACG_simulations_main_execution.py; this script imports Circuit_design/agents.py, Circuit_design/environment.py, Circuit_design/layers.py, and Circuit_design/plotting_functions.py
Fig. 8e: Circuit_design/striosome_dopamine_control.py
Fig. 8g: Circuit_design/d1_d2_both_necessary.py
Sup. Fig. 2a: Associative_learning_correlations/theoretical_2D_curve.py
Sup. Fig. 2b: Associative_learning_correlations/novelty_colormap.py
Sup. Fig. 2d: Associative_learning_correlations/novelty_colormap.foraging_with_large_costs_RPE_pIG_relationship.py
Sup. Fig. 2e-g: Associative_learning_correlations/open_gym_comparison.py
Sup. Fig. 3: Ramping_and_state_value/dopamine_and_state_value_correlation.py
Sup. Fig. 4d: Multi_cue_learning/chunking_vs_TD_learning_blocking_task.py
Sup. Fig. 4f,g,i,j: Multi_cue_learning/chunking_vs_TD_learning_two_cue_task.py
Sup. Fig. 5: Action_initiation/clusters_of_brackets.py
Sup. Fig. 6: Action_initiation/sensivity_by_baseline_policy_IG.py
Sup. Fig. 7: Decision_making_biases/decision_making_manifold.py
Sup. Fig. 8d,e: Distribution_across_neurons/distribution_across_neurons.py
Sup. Fig. 9c: Disorders/conceptual_disorder_policy_IG_axis.py
Sup. Fig. 10c-e: Circuit_design/ACG_simulations_main_execution.py; this script imports Circuit_design/agents.py, Circuit_design/environment.py, Circuit_design/layers.py, and Circuit_design/plotting_functions.py
Sup. Fig. 11b: Circuit_design/striosome_dopamine_control.py
Sup. Fig. 12b,c: Circuit_design/d1_d2_both_necessary.py
Sup. Fig. 13b-f: Naturalistic_foraging/naturalistic_rodent_foraging.py