In this study, we present a machine learning method for visualizing subjective images in the human mind based on brain activity. Although many previous studies have demonstrated that images observed by humans can be reconstructed from their brain activity, the visualization (externalization) of mental imagery remains a challenge. To achieve this long-standing milestone, we combined a previous visual image reconstruction method with recently developed neural network technology into a single Bayesian estimation framework. The results demonstrated that our proposed framework successfully reconstructed both seen and imagined images from brain activity, which would provide a unique tool for directly investigating the subjective contents of the brain such as illusions, hallucinations, and dreams.
A Python implementation of our proposed algorithm will be made available soon after the publication of the journal version of this article.
-
Naoko Koide-Majima, Shinji Nishimoto, and Kei Majima.
"Mental image reconstruction from human brain activity:
Neural decoding of mental imagery via deep neural network-based Bayesian estimation."
Neural Networks (2023).
https://doi.org/10.1016/j.neunet.2023.11.024
@article{koide2024mental,
title={Mental image reconstruction from human brain activity: Neural decoding of mental imagery via deep neural network-based Bayesian estimation},
author={Naoko Koide-Majima and Shinji Nishimoto and Kei Majima},
booktitle={Neural Networks},
pages={349--363},
month={Feb},
year={2024},
doi={https://doi.org/10.1016/j.neunet.2023.11.024}
}