Attention-based Deep Multiple Instance Learning could be applied in a wide range of medical imaging applications. Supported by the project "Deep Learning for Survival Prediction"@UTA-SMILE, I wrote the Keras version of ICML 2018 paper "Attention-based Deep Multiple Instance Learning" (https://arxiv.org/pdf/1802.04712.pdf) in this repo to share the solution for Keras users.
The official Pytorch implementation can be found here. I built it with Keras using Tensorflow backend. I wrote attention layers described in the paper and did experiments in colon images with 10-fold cross validation. I got the very close average accuracy described in the paper and visualization results can be seen as below. Parts of codes are from https://github.com/yanyongluan/MINNs.
When train the model, we only use the image-level label (0 or 1 to see if it is a cancer image). The attention layer can provide an interpretation of the decision by presenting only a small subset of positive patches.
- Colon cancer dataset [Data]
- Processed patches [Google Drive]
I put my processed data here and you can also set up according to the paper. If you have any problem, please feel free to contact me.
| Year | Author list | Title | Conference/Journal |
|---|---|---|---|
| 2020 | Jiawen Yao, Xinliang Zhu, Jitendra Jonnagaddala, Nicholas Hawkins, Junzhou Huang | Whole slide images based cancer survival prediction using attention guided deep multiple instance learning networks. [Pytorch] | Medical Image Analysis, 101789, 2020, [PDF], [arxiv] |
Other important work used multiple-instance learning in medical imaging include (list will be updated frequently)
| Year | Author list | Title | Conference/Journal |
|---|---|---|---|
| 2021 | Ming Y. Lu, Tiffany Y. Chen, Drew F. K. Williamson, Melissa Zhao, Maha Shady, Jana Lipkova & Faisal Mahmood | AI-based pathology predicts origins for cancers of unknown primary. [Pytorch] | Nature, arxiv |
| 2021 | Ming Y. Lu, Drew F. K. Williamson, Tiffany Y. Chen, Richard J. Chen, Matteo Barbieri & Faisal Mahmood | Data-efficient and weakly supervised computational pathology on whole-slide images. [Pytorch] | Nature Biomedical Engineering, arxiv |
| 2021 | Jianan Chen, Helen M. C. Cheung, Laurent Milot and Anne L. Martel | AMINN: Autoencoder-based Multiple Instance Neural Network Improves Outcome Prediction of Multifocal Liver Metastases. [Keras] | MICCAI 2021 arxiv |
| 2020 | Ole-Johan Skrede et al. | Deep learning for prediction of colorectal cancer outcome: a discovery and validation study | Lancet |
| 2019 | Shujun Wang, Yaxi Zhu, et al. | RMDL: Recalibrated multi-instance deep learning for whole slide gastric image classification [Keras] | Medical Image Analysis arxiv |
If you have any questions about this code, I am happy to answer your issues or emails (to yjiaweneecs@gmail.com).
I plan to review recent work using Deep MIL techniques in medical imaging and Your suggestions are very welcome !
The work conducted by Jiawen Yao was funded by Grants from the UTA-SMILE Lab.