This folder contains all the material that were originally presented in a two-hour FLUX 2021 Model-free and model-based computational workshop on September 17 2021. Since then, these exercises have been made publicly available. Please reach out to us if you catch any errors in these scripts!
Created by Claire Smid and Wouter Kool
This repository contains three sets of exercises that guide you through simulating model-free and model-based reinforcement learning agents on a two-step task, and then lets you fit this simulated data to computational model. If you are not familiar with Github, the easiest way to work with these files is to click on the green button with 'Code' in it on the top right, and then clicking on 'Download ZIP' in the drop-down menu to download these scripts to your computer, where you can unzip them. We would recommend for this workshop that you download the files to your desktop, so you have them easily accessible. We would also recommend you use Google Chrome, especially if using Google Colab.
The exercises are built to be worked through the scripts chronologically, so start with reading Step1_ModelFreeRL.
If you want to work with Google Colab, the easiest way is to then upload the unzipped folder to your Google Drive (https://drive.google.com/drive/my-drive). Once this is done, you can open the script in Google colab just by double-clicking on a script!
If you want to work with Jupyter Notebook instead, the easiest way is to use Anaconda (https://www.anaconda.com/) which is like a package manager for Python based applications. If you launch Jupyter notebook from there, it will open in your browser, and you can navigate to the downloaded scripts on your computer inside Jupyter Notebook and open them from there.
You can of course also open the scripts in other Python applications (e.g. PyCharm, Spyder), however, we would recommend using either Google Colab or Jupyter Notebook so you can work with interactive cells.
When you do the exercises we would recommend that you copy the exercise script, just so that you can have a backup version to refer back to if you get stuck during the exercise. You can easily to this in Google Colab by clicking on 'File' and then 'Save a copy in drive' which will open up a new tab with a copy of the script. In Jupyter Notebook, you can click on 'File' and then 'Make a Copy'.
The folder "solutions" contains the solutions to all the exercises in the "exercises" folder. Of course, you can spoil yourself, and go straight to the solutions. However, we encourage you to give the exercises a try. You might get stuck, but figuring out the solutions through some struggle (and some Googling) will greatly help you understand how this stuff works.