| title | date | layout |
|---|---|---|
Reviewer Guidelines |
03-14-2015 |
blank |
Reviewing for the Programming Historian is a great way to learn new technical skills (at any level) and engage with the digital humanities community at the same time. We go out of our way to make sure our reviewers get ample credit and recognition for their work. Because reviewers directly contribute to significantly improving lessons, you can take pride in how your work helps our many thousands of users.
Our peer review process is a bit different from what might be considered the "traditional" peer review process. We do not solicit reviews to judge whether a tutorial is "good enough" to be published. Rather, we consider the review process an integral component of a collaborative, productive, and sustainable effort for scholars to create useful technical resources for each other. Once a tutorial slips into our editorial workflow, our goal is to do everything we can to make sure the tutorial becomes as useful as possible and published in a reasonable amount of time.
To review a tutorial is not simply to assess its merits and flaws (though it is important to do these as well), or judge whether it's appropriate for Programming Historian, but to help improve the tutorial as an outside reader. We ask that reviewers comment on tutorials as they would on a paper or similar for a colleague--to provide fair and constructive criticism (and of course praise) that will improve the tutorial for a wide range of readers. We don't send anything out for review that isn't within a reasonable distance of publishable quality. A few extra perspectives and suggestions (beyond those of the editor) greatly improve lessons and make them of great (and unique) value to the community.
We highly value transparency in our lesson production and review process. Our review process happens at Github, where our lessons are hosted. Namely, we use Github's commenting and issue tracking capabilities to keep track of suggestions and how they are handled and discussed by the authors and reviewers. This helps avoid important conversations becoming lost in archived email threads. Future tutorial users can easily rekindle a old conversation that has become newly relevant.
Therefore, your work as a reviewer--and your identity--will be fully visible to the author. Comments should engage with the author and lesson directly, rather than the review editor. If you as a reviewer would like to share some comments privately with the reviewing editor, you are welcome to email those separately, but we aim for as much transparency as possible.
Unless you instruct us otherwise, your name will be indicated as a reviewer on the lesson's page at Programming Historian when it is officially published; you'll also be listed on our contributors page. These are, of course, insufficient tokens of appreciation, but we want to emphasize that Programming Historian is the work of volunteers and give credit where credit is due.
The informal style of Programming Historian lessons can make the lessons appear deceptively simple to write. In fact, writing a good tutorial is every bit as demanding, if not more so, than any other kind of scholarly writing. Below are a few common questions to keep in mind as you review a lesson. Some will be more relevant than others, depending on the topic, intended audience, and the difficulty of the tutorial. Needless to say, this is neither a restrictive or comprehensive list; we do not ask that reviewers respond to each and every question, but hopefully they can provide some general guidance.
If you haven't dabbled much at the Programming Historian already, you'll see that there is no standard audience across all the lessons. Some are for complete beginners, others are for those considerably more comfortable with technical concepts and methods, and yet others are directed toward experienced digital historians looking to solve difficult technical challenges. We welcome such variety!
While we like each of our lessons to take the unique voice of its author, we also want each individual lesson to remain clear and consistent in its tone. More specifically, we want the technical explanations (and difficulty) to remain as constant as possible throughout the lesson. As a reviewer, it is quite useful to note any sections of a tutorial directed at more technically advanced users that seem to over-explain relatively simple concepts; the converse is true, too: we want to avoid sections of tutorials aimed at beginners that don't adequately explain fundamental concepts central to the tutorial. Some questions to keep in mind:
- Does the author address a consistent model reader throughout the lesson?
- Are some concepts or steps over-explained while other are under-explained?
- Does the audience seem to match at least vaguely with other Programming Historian lessons? How is it new?
- What software / programming languages are required?
- What prerequisite skills are needed?
- What familiarity or experience is needed?
- What data is needed? Is it readily available?
- Are there clearly defined learning objectives or set of skills near the top of the lesson?
- Are there useful secondary skills to be gained / practiced from the lesson?
- Do screenshots and other diagrams illustrate crucial steps / points of the lesson?
- Do sections and section headings provide clear signage to the reader?
- Does the tutorial suggest why the explained tools or techniques are useful in a general way?
- Does the tutorial suggest how a reader could apply the concepts (if not concrete steps) of the lesson to their own work?
- Should a long lesson be divided into smaller lessons?
- Are there logical stopping points throughout the lesson?
- If data is required, it is available to download at various points throughout the lesson (or different versions of it as the tutorial may require)?
- Does the lesson build upon an existing lesson and explain how?
- Does the lesson tie into existing lessons and have appropriate links?
We manage all our peer review comments through the issue tracker on Github, so all reviewer feedback will take the form of a comment on a Github "issue". It sounds more complicated than it really is. Please follow the directions for reviewing lessons with GitHub.