Peer Review #2

[distillpub-reviewers]

The following peer review was solicited as part of the Distill review process.

The reviewer chose to waive anonymity. Distill offers reviewers a choice between anonymous review and offering reviews under their name. Non-anonymous review allows reviewers to get credit for the service they offer to the community.

Distill is grateful to Patricia Robinson for taking the time to review this article.

---

General Comments

Overall a pleasure to review. Caught a few simple errors that might be worth changing. ie "Which graph attributes we update and in which order we update them is one design design behind GNNs." (change to: one design) Might be good to run through an editor to catch these spelling and grammar. :). My biggest ask would be adding in my engaging visuals if possible. Wonderful work. I learned a lot.

---

Distill employs a reviewer worksheet as a help for reviewers.

The first three parts of this worksheet ask reviewers to rate a submission along certain dimensions on a scale from 1 to 5. While the scale meaning is consistently "higher is better", please read the explanations for our expectations for each score—we do not expect even exceptionally good papers to receive a perfect score in every category, and expect most papers to be around a 3 in most categories.

Any concerns or conflicts of interest that you are aware of?: No known conflicts of interest
What type of contributions does this article make?: Exposition on an emerging research direction

Advancing the Dialogue	Score
How significant are these contributions?	4/5

Outstanding Communication	Score
Article Structure	5/5
Writing Style	5/5
Diagram & Interface Style	3/5
Impact of diagrams / interfaces / tools for thought?	3/5
Readability	5/5

Comments on Readability

Language is really the strength in this piece. Authors take readers on a delightful journey. Truly fun to read. Visuals fell a little short. Would have been engaged to see more interactive moments. Aesthetics aside, I think more compelling diagrams have the potential to help first timers learn in more dynamic and intuitive ways.

Scientific Correctness & Integrity	Score
Are claims in the article well supported?	5/5
Does the article critically evaluate its limitations? How easily would a lay person understand them?	5/5
How easy would it be to replicate (or falsify) the results?	5/5
Does the article cite relevant work?	5/5
Does the article exhibit strong intellectual honesty and scientific hygiene?	5/5

Comments on Scientific Integrity

Enjoyed the balance of citations that are more recent (5 years) and important foundational works (circa 1969 and 1976). Found sources very trustworthy. Reproducibility seems fairly simple.

[beangoben]

We thank the reviewer for their time and attention, we have taken their comments into consideration and we think our work is stronger because of them.

Next, we summarize most of the changes that we have made based on feedback from all reviewers:

Reviewer 1 made several points on improving the writing and presentation of ideas, this resulted in simplifying the language for several sentences, breaking down paragraphs and expanding examples for some concepts.

Reviewer 1 also asked us to improve on the "lessons" of the GNN playground. These lessons became the subsection ""Some empirical GNN design lessons" which details new interactive visualizations that show some of the larger architecture trends for the playground.

Reviewer 3 made a point about expanding on the connection between Transformers and also on some of the current limitations with GNNs and message passing frameworks.

All reviewers noted a few typos, latex equations errors and grammatical mistakes that we have fixed. The bibliography has expanded slightly.

For a more detailed breakdown of the changes:

• [Reviewer 1] Broke up the first paragraph and added more explicit examples.
• [Reviewer 1] Changed ordering of appearance of graph attributes.
• [Reviewer 1] Add a sentence to the first paragraph of "Graphs and where to find them" to better introduce and motivate the reader on why we look at images and text as graphs.
• [Reviewer 1 & 3]For the "Text as graphs" section, we clarified the caption for the figure. Made a connection to Transformers.
• [Reviewer 1] In the "Graph-valued data in the wild" section, for the other examples table we added a Domain column for each graph to denote the area of the dataset.
• [Reviewer 1] When introducing graphs we have added an additional visualization that showcases embeddings for different graph attributes. Clarified writing when showing scalar values and that in practice we expect vector values.
• [Reviewer 1] Added an aside to each example in the "Passing messages between parts of the graph" section.
• [Reviewer 3] Expanded caption in figure for "comparing aggregation operations" section.
• [Reviewer 3] Expanded the connection between transformers and GNNs in the Graph Attention Networks section.
• [Reviewer 1 & 3] Added "Some empirical GNN design lessons", which has 5 interactive plots with an insight per plot.
• [Reviewer 1] Added a section on more detailed notes on implementing graph convolutions.
• [Reviewer 1] Expanded on some of the current limitations of GNNs and areas for improvement on modelling.