Reviews

This directory should contain all reviewer feedback from submitted papers to peer-reviewed venues. This should include both accepted and rejected papers. Keep this directory organised with subdirectories using the publication key format (e.g., reviews/icse-2020).

Files within these subdirectories should include the following:

• A PDF copy of the manuscript you submitted to the venue.
• A copy of the notification email received (read more below).
• A breakdown of the reviewer's feedback in a spreadsheet (read more below).
• A rebuttal letter (as a Google Doc), if required.

Working Through Reviewer's Concerns

For archival and record-keeping purposes, store a copy of the notification email you receive from the venue in the publication's subdirectory. This can be done by creating an .eml file from the email. To create an .eml file from Outlook for Mac or Mail for Mac, simply drag the email from your inbox to wherever you'd like to store it.

Reading an entire reviewer's notification email can be daunting and overwhelming. Fret not! Deciphering this information within a spreadsheet makes it far easier for you to address reviewer's concerns. A template of such a spreadsheet is available here.

A Workflow for Going Through Feedback

Here is a workflow you can follow in conjunction with the spreadsheet template:

1. Start by opening up the notification email.
2. Go through the first reviewer's comments (R1), and systematically go top to bottom from their review.
3. Pick apart only criticisms about your paper from the reviewers (i.e., you can ignore positive feedback to focus on what you need to address) and extract each as a 'quote' from their review.
4. Add this quote into the "Reviewer's Quotes" column.
5. Assign the quote a comment number, ensure the reviewer number is assigned (R1 for Reviewer 1, R2 for Reviewer 2 etc.)
6. Propose your own resolution to action the comment (in the Proposed Resolution column) using a traffic-light system:

• Green = I know (roughly) what to do to action the reviewer's comment, but would be good to get my co-author's/supervisor's thoughts on this
• Yellow = I think I have an idea what to do to action the reviewer's comment, but I am not sure. I want my co-author's/supervisor's to share their thoughts on how they'd action the reviewer's comment.
• Green = I am completely lost and need help. Seeking support from co-authors/supervisors on what to do here.

1. Rinse and repeat steps 3–6 for reviewer 2 (R2) and reviewer 3 (R3) (and so on).
2. Once finished, go through all the comments and assign a 'theme' to them (since some reviewers will loosely discuss similar topics about the paper and it's easier to see all their quotes about the same topic in one go). E.g., some reviewers will talk about the scope of the paper, or the survey study in a paper, or the related work sections (and so on).
3. Sort the spreadsheet by the "Theme" column, and get your co-authors/supervisors to look at the spreadsheet and provide feedback on my proposed resolution. (Use their initials under the relevant column headers; e.g., in the template RV = Rajesh Vasa; JG = John Grundy; AC = Andrew Cain.)
4. Once everyone has chipped in some comments, book a meeting to discuss with everyone in the room. Go through each comment to see if anyone has any additional feedback.
5. Once your discussion is over, go through the paper and work through each concern one-by-one. Check off using the "Actioned?" column.
6. Get your coauthors to review the changes until everyone is happy with it, or else repeat steps 10 and 11.
7. Resubmit your paper.
8. Get accepted, or else go back to step 1 ;-)