Review #1 #6
The following peer review was solicited as part of the Distill review process.
The reviewer chose to keep anonymity. Distill offers reviewers a choice between anonymous review and offering reviews under their name. Non-anonymous review allows reviewers to get credit for the service they offer to the community.
This paper combined several techniques (dimensionality reduction) to inspect the visual features learned in a reinforcement learning (RL) model. The main hypothesis made in the paper is the diversity hypothesis: if the RL models are trained with more diverse environments, the RL models will become more interpretable. To support this hypothesis, this paper used the procedurally-generated video game environment CoinRun as the research platform and took advantage of feature visualization techniques to identify visually interpretable focal points of the model. The interfaces provided in the paper have a rich collection of visual examples on different games trained with different experimental settings, which well support the hypothesis and are very helpful to help readers understand the claim.
This paper is well-written and provides many visual examples to help explain the ideas. Below are some suggestions on improving the writing:
Overall, I think this paper provides a valuable method and example of understanding visual features learned in an RL model and their interpretability. It’s a valuable contribution to the RL community.
Distill employs a reviewer worksheet as a help for reviewers.
The first three parts of this worksheet ask reviewers to rate a submission along certain dimensions on a scale from 1 to 5. While the scale meaning is consistently "higher is better", please read the explanations for our expectations for each score—we do not expect even exceptionally good papers to receive a perfect score in every category, and expect most papers to be around a 3 in most categories.
Any concerns or conflicts of interest that you are aware of?: No known conflicts of interest
The text was updated successfully, but these errors were encountered:
We are extremely grateful to the reviewer for their thoughtful comments. We have made a number of changes thanks to their suggestions.