Mental image reconstruction

This is a gitHub repository for our recent work "Mental image reconstruction from human brain activity: Neural decoding of mental imagery via deep neural network-based Bayesian estimation."

• Project page on this work is here.

• Demo code on Google Colab is here. You can reproduce our results without any cumbersome installation.

Abstract

In this study, we present a machine learning method for visualizing subjective images in the human mind based on brain activity. Although many previous studies have demonstrated that images observed by humans can be reconstructed from their brain activity, the visualization (externalization) of mental imagery remains a challenge. To achieve this long-standing milestone, we combined a previous visual image reconstruction method with recently developed neural network technology into a single Bayesian estimation framework. The results demonstrated that our proposed framework successfully reconstructed both seen and imagined images from brain activity, which would provide a unique tool for directly investigating the subjective contents of the brain such as illusions, hallucinations, and dreams.

Paper

• Naoko Koide-Majima, Shinji Nishimoto, and Kei Majima.
  "Mental image reconstruction from human brain activity:
  Neural decoding of mental imagery via deep neural network-based Bayesian estimation."
  Neural Networks (2024).
  https://doi.org/10.1016/j.neunet.2023.11.024

BibTeX

@article{koide2024mental,
  title={Mental image reconstruction from human brain activity: Neural decoding of mental imagery via deep neural network-based Bayesian estimation},
  author={Naoko Koide-Majima and Shinji Nishimoto and Kei Majima},
  booktitle={Neural Networks},
  pages={349--363},
  month={Feb},
  year={2024},
  doi={https://doi.org/10.1016/j.neunet.2023.11.024}
}