Skip to content

Hacking the human visual cortex (brain decoding).

Notifications You must be signed in to change notification settings

syrkis/neuroscope

Repository files navigation

A Multi-Modal Exploration of the Algonauts Dataset

This repository explores the connection between visual stimuli and brain response. It is based on the Algonauts dataset, which contains fMRI data from 8 subjects, each reacting to thousands of images. The Algonauts dataset, in turn is based on the the NSD dataset, which uses images from COCO. COCO image category, supercategory, and captions are used in this project, in addition to the pure Algonauts data.

Tools of note include JAX and nilearn, in addition to the usual suspects (numpy, pandas, matplotlib, etc.). src contains the main code, notebooks contains the main notebooks, and data contains the data (which is not included in this repository). src is further divided into data, models, train, and utils for data loading, model definitions, training, and utility functions, respectively.

To run the project locally, please ensure that data is populated with the Algonauts data, and our (currently unavailable) COCO augmentations. Then, run pip install -r requirements.txt to install the necessary dependencies in a virtual python 3.11 environment of your choice. Finally, run jupyter lab to start the notebook server.

Training logs are available at https://wandb.ai/syrkis/neuroscope

About

Hacking the human visual cortex (brain decoding).

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published