The following peer review was solicited as part of the Distill review process. The review was formatted by the editor to help with readability.
The reviewer chose to keep anonymity. Distill offers reviewers a choice between anonymous review and offering reviews under their name. Non-anonymous review allows reviewers to get credit for the service them offer to the community.
Distill is grateful to the reviewer for taking the time to write such a thorough review.
Most of my concerns have been posted in the "Communication" feedback section under general comments. Please see there for a longer review. I encourage the authors to continue this work and polish the current draft.
The first three parts of this worksheet ask reviewers to rate a submission along certain dimensions on a scale from 1 to 5. While the scale meaning is consistently "higher is better", please read the explanations for our expectations for each score—we do not expect even exceptionally good papers to receive a perfect score in every category, and expect most papers to be around a 3 in most categories.
Any concerns or conflicts of interest that you are aware of?: n/a What type of contributions does this article make?: Novel results
Advancing the Dialogue
How significant are these contributions?
Diagram & Interface Style
Impact of diagrams / interfaces / tools for thought?
Remarks on communication
Below I first list some pros of the communication of this submission, followed by a list of concerns and a general review.
The first interactive graphic and text in the Background is well motivated and interesting to read
The description and motivation of the article using the Data-visual correspondence is a great framing of the visualization techniques used here, especially when applied to machine learning
After the Grand Tour technique is introduced and explored, the adversarial example exploration was compelling to see other uses of the developed visualization design
Major: Submission readability
While the submission is readable and the main ideas are accurately presented, it suffers from pervasive grammatical errors and uncommon word choices that negatively impact the communication of the research. This is my primary concern with the submission. The research here is interesting and I am excited to see it eventually published, but as is, it needs multiple rounds of prose iteration and improvement to meet academic publishing standards (Distill, or any other computer science conference).
Major: Grand Tour technique introduction
Given the emphasis on the Grand Tour technique, which is an interesting perspective on visualizing neural network activations, there is likely a better way to introduce and frame the technique with respect to modern techniques. Specifically, the phrase “somewhat-forgotten” feels self-defeating and could be improved to excite the reader about this technique. Furthermore, for readers that may not know of it, was it somewhat-forgotten for good reason? Or did it simply fall out of research popularity? Some discussion around this may be helpful to properly situate the technique in history.
Major: Better presentation of technical details
The technical details section would be better placed in the acknowledgements of the article. However, since it is quite long (somewhere between 1/4 and 1/3 of the article), another idea is that it could be highlighted with a banner description and hidden by default with an indicator to reveal upon a reader interaction (e.g., show/hide button).
Minor: Hero / banner interactive graphic
While the top-level graphic is eye-catching and hints at the techniques used in the paper, perhaps a caption and light annotation could help improve its message to readers. This would help push it towards substantial preview of the article, rather than only a nice looking animation.
Minor: Broken graphic
“The Grand Tour of the softmax layer” figure was broken: the image/points drop down menu did not update the visualization correctly, nor did the data instance positions did not update.
“The math behind the two is simple.” Be careful with phrases such as these, since readers will come from many diverse backgrounds with different levels of mathematical fluency.
Adding a zoom slider to all Grand Tour visualizations (not just the later ones) could be useful
“vis” instead of “visualization,” although explicitly indicated in the submission, reads a bit informal
Scientific Correctness & Integrity
Are claims in the article well supported?
Does the article critically evaluate its limitations? How easily would a lay person understand them?
How easy would it be to replicate (or falsify) the results?
Does the article cite relevant work?
Does the article exhibit strong intellectual honesty and scientific hygiene?
Comments on Scientific Correctness and Integrity
While there could be other citations to include to support the first paragraph of the introduction (e.g., work related to visualization and visual analytics in deep learning, neural network interpretability), the current text passes.
In the Discussion, I expected to see citations or relevant links/materials to corroborate “The trade-offs between small multiples and animations is an ongoing discussion in the vis community.”
The text was updated successfully, but these errors were encountered: