This repository explores the connection between visual stimuli and brain response. It is based on the Algonauts dataset, which contains fMRI data from 8 subjects, each reacting to thousands of images. The Algonauts dataset, in turn is based on the the NSD dataset, which uses images from COCO. COCO image category, supercategory, and captions are used in this project, in addition to the pure Algonauts data.
Tools of note include JAX and nilearn, in addition to the usual suspects (numpy, pandas, matplotlib, etc.).
src
contains the main code, notebooks
contains the main notebooks, and data
contains the data
(which is not included in this repository). src
is further divided into data
, models
, train
, and utils
for data loading, model definitions, training, and utility functions, respectively.
To run the project locally, please ensure that data
is populated with the Algonauts data, and our (currently unavailable) COCO augmentations.
Then, run pip install -r requirements.txt
to install the necessary dependencies in a virtual python 3.11 environment of your choice.
Finally, run jupyter lab
to start the notebook server.
Training logs are available at https://wandb.ai/syrkis/neuroscope