Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
The following peer review was solicited as part of the Distill review process.
The reviewer chose to waive anonymity. Distill offers reviewers a choice between anonymous review and offering reviews under their name. Non-anonymous review allows reviewers to get credit for the service they offer to the community.
Distill is grateful to Ruth Fong for taking the time to review this article.
Small suggestions to improve readability:
Main weakness regarding ""Scientific Correctness & Integrity"" is a lacking discussion about related works and limitations:
Distill employs a reviewer worksheet as a help for reviewers.
The first three parts of this worksheet ask reviewers to rate a submission along certain dimensions on a scale from 1 to 5. While the scale meaning is consistently "higher is better", please read the explanations for our expectations for each score—we do not expect even exceptionally good papers to receive a perfect score in every category, and expect most papers to be around a 3 in most categories.
Any concerns or conflicts of interest that you are aware of?: No known conflicts of interest
Thank you for the detailed comments! Based on your feedback, we’ve made some changes to the article and added several new sections. In particular:
I agree. In the name of brevity, the first draft of this article failed to cite several saliency/interpretability methods that were worth citing. In our latest draft we significantly expanded the list of articles we cite. We still don’t discuss most non-path methods in detail, as we wanted not to broaden the scope of the article too much. With that said, we added a new section “Expectations, and Connections to SmoothGrad” that discusses the formal connection between SmoothGrad and expected gradients. Throughout the article, we also discuss the concept of a “baseline” input in more generality than just in the context of path attribution methods.
Good point. We now do briefly mention this in the section “The Pitfalls of Ablation Tests.” We also mention that there is a broad discussion about whether or not we should present images outside the training domain to our models that goes beyond the scope of our original article.
This another really good point. Despite being an article about baselines, our original article only presented two different baseline choices. Based on this feedback, we decided to add two new sections: “Alternative Baseline Choices” and “Averaging Over Multiple Baselines” that covers existing ideas about baselines. In particular, we discuss and visualize random noise and blurred input as baselines. We now present over 6 different baselines and several variants thereof, and hope that this provides a better and more nuanced picture of possible baselines.
I agree that it is important to discuss how to evaluate interpretability methods in our article, which is something our first draft omitted. Our new section “Comparing Saliency Methods” does this. Although it doesn’t comprehensively evaluate all of the baselines we present, we hope that it provides some relevant discussion in this area. We don’t run more comprehensive evaluations on our baselines mostly for computational reasons: there are many different baselines, hyper-parameters and evaluation metrics to compare across and we wanted to keep the main focus on the assumptions behind each baseline rather than a quantitative assessment of them.
I tried to do this in our new draft. We have some additional discussion on the completeness axiom right after the third figure.
Unfortunately, I didn’t address either of these points. I didn’t address the former because when I tried adding additional examples to the figure, they got quite cluttered and took a while to load. I haven’t addressed the latter simply because I felt the other points were more important to address. I will make an effort to add a collab for at least one figure in the future.
Overall, I hope that we’ve managed to address at least some of your main concerns: especially those regarding omissions with respect to existing ideas in literature. We’ve made significant efforts to better place this article around existing ideas, and will continue to do so based on future feedback.