Skip to content

Visualizing 3D ResNet for Medical Image Classification With Score-CAM

Notifications You must be signed in to change notification settings

reshalfahsi/3d-viz-score-cam

Repository files navigation

Visualizing 3D ResNet for Medical Image Classification With Score-CAM

colab
architecture Live interaction with the 3D Score-CAM result.

Intepretability is one of the concerns regarding the application of AI, or, to be exact, deep learning, in the medical field, especially medical image recognition. In a venture seeking to explain what is going on or what the network perceives computationally, one can leverage the class activation map (CAM) of the model. Score-CAM, one of the CAM variants, breakthroughs the preceding CAM methods by dropping the reliance on gradients. Instead, it benefits the full potential of the forward propagation of the model, running inference by normalized-weighting via element-wise product with the input. Then, the output logit of the target category is combined with the CAM acquired before to get the final outcome. In this project, the 3D version of ResNet is employed. To evaluate the aforementioned methods, the OrganMNIST3D dataset of MedMNIST is used. The deletion area under the curve (DAUC) and the insertion area under the curve (IAUC) are adopted to measure the performance of Score-CAM.

Experiment

Catch up on this link to comprehend the training, testing, and visualization pertaining to this project.

Result

Image Classification

Quantitative Result

The table below exhibits the outcome, quantitatively.

Test Metric Score
Loss 0.570
Accuracy 89.79%

Accuracy and Loss Curve

acc_curve
Accuracy curves of 3D ResNet on the train and validation sets.

loss_curve
Loss curves of 3D ResNet on the train and validation sets.

Visualization

Quantitative Result

Overall DAUC and IAUC scores of the Score-CAM on the 4th layer of 3D ResNet:

Test Metric Score
DAUC 0.2576 ± 0.1710
IAUC 0.6750 ± 0.1944

Qualitative Result

The following are snapshots of the individual DAUC and IAUC scores and their Score-CAM outcomes.

bladder
The result of the bladder.

heart
The result of the heart.

kidney-left
The result of the left kidney.

kidney-left
The result of the right kidney.

lung-left
The result of the left lung.

kidney-left
The result of the right lung.

spleen
The result of the spleen.

Citation

To cite this repository:

@misc{3dviz-scorecam,
   title = {Visualizing 3D ResNet for Medical Image Classification With Score-CAM},
   url = {https://github.com/reshalfahsi/3d-viz-score-cam},
   author = {Resha Dwika Hefni Al-Fahsi},
}

Credit