This is code used to perform the VAE analyses in:
Singh Alvarado, J., Goffinet, J., Michael, V. et al. Neural dynamics underlying birdsong practice and performance. Nature (2021). https://doi.org/10.1038/s41586-021-04004-1
The main files are ssl/poe_finch.py, which contains the product of experts
joint-encoding VAE, and ssl/cca_finch.py, which contains the baseline
separate-encoding model. Additionally, finch_exp.py contains code for testing
model performance, neural_axis_two_color_plots.py contains code for
visualizing the shared information found by the models, and detrend_specs.py
contains code for remove time-of-day trends from song and neural activity.
If you are interested in applying a joint-encoding VAE to your own data, see the PoE VAE repo for more flexible implementations of various multimodal VAEs with better documentation.