Contributing to the Guidelines

Matthew Kay edited this page Jan 5, 2018 · 16 revisions

This is a draft of the proposal and review process for the transparent statistics guidelines, to be updated as we develop the process alongside our first set of chapters. Feedback and suggestions for refining this process are welcome!

We welcome contribution at all levels. To make it easy to "dive in", we have organized this document in increasing order of workload from the contributor's perspective. This document describes the mechanics of how to contribute to the guidelines. See the Style Guide for a description of what the guidelines should look like.

The Transparent Statistics Guidelines are organized into chapters. Chapter 1 lays out general guiding principles for transparent statistics. The remaining chapters are topics (e.g., Chapter 2 is on "Effect Size"). Each topic contains an FAQ on that topic and one or more exemplars.

Endorsing a chapter

If you like a particular chapter and do not find anything to comment on it (or if you find that your comments have been properly taken into account), you can choose to endorse it. This endorsement process is important. The more a transparent statistics chapter is endorsed, the more weight it carries for the rest of the CHI community.

To endorse a chapter: (TODO)

Commenting on or suggesting changes to an existing chapter

We welcome any sort of feedback on the existing chapters. The easiest way to "dive in" is to open an issue on Github, comment on an existing issue, or comment on a pull request. We value feedback, so if you are not sure where to send your feedback, open an issue!

Proposing minor changes to an existing chapter

Minor changes are those that do not need broader review. These might be minor changes to style or formatting for clarity, but should not be substantial changes to content. They may also include adding a citation to an existing statement that does not change the content of the statement (merely gives it more support).

To propose minor changes to an existing chapter:

  1. Make those changes on your own branch or fork.

  2. Make a pull request to merge those changes onto master.

  3. Pull request is accepted by a core member (this may be the same person who is making the minor change).

Proposing major changes to an existing chapter

Major changes are any that change meaning or add new content (e.g. a new exemplar in an existing topic).

  1. Create initial changes in RMarkdown/git. New content/modifications should be created on your own branch or fork.

  2. Open pull request to track review. Authors create a pull request to merge their changes onto the master branch. The pull request should be titled something like Changes to [chapter] chapter: [X, Y, Z] and have the review label, and should have two reviewers with subject matter expertise who were not involved in the creation of the new content assigned to it. This checklist should be the contents of the first comment in the pull request:

    - [ ] First review by another community member *with* subject matter expertise
        - [ ] Testing not necessary
    - [ ] Second review by another community member *with* subject matter expertise
        - [ ] Testing not necessary
    - [ ] Check & fix coding style
    - [ ] First test by another community member *without* subject matter expertise
    - [ ] Second test by another community member *without* subject matter expertise
    

    Coding style: see Style-Guide

  3. Update pull request as review proceeds. As the above checklist is completed, the initial comment should be edited to tick those boxes like so:

    - [x] First review by another community member *with* subject matter expertise
        - [ ] Testing not necessary
    - [ ] Second review by another community member *with* subject matter expertise
        - [ ] Testing not necessary
    - [ ] Check & fix coding style
    - [ ] First test by another community member *without* subject matter expertise
    - [ ] Second test by another community member *without* subject matter expertise
    

    If the new content is of a type that does not require testing, during the first two steps of review, a reviewer can check their corresponding "Testing not necessary" box. If both such boxes are checked, the last two steps (testing) can be skipped. In general, all new exemplars should be tested, but other types of changes (e.g. new FAQ content) may not require testing. This is indicated by checking those boxes and striking out the test steps:

    - [x] First review by another community member *with* subject matter expertise
        - [x] Testing not necessary
    - [x] Second review by another community member *with* subject matter expertise
        - [x] Testing not necessary
    - [ ] Check & fix coding style
    - [ ] ~~First test by another community member *without* subject matter expertise~~
    - [ ] ~~Second test by another community member *without* subject matter expertise~~
    

    All comments and discussion about the chapter should be made on this pull request so that everyone involved in the review is notified and so that relevant comments and changes are documented.

  4. Accept request to merge new content onto master. Once the above reviews are complete and all boxes checked (or struck out), the pull request is accepted by a guidelines-core member who is not one of the primary chapter authors.

Developing a new topic

To develop a new topic:

  1. Create Google docs initial version. FAQ started in Google docs and developed among initial authors. Place each FAQ as one file in the FAQ folder on the shared Google Drive. A placeholder RMarkdown file can also be added to Github, pointing to the corresponding Google doc using a block like this:

    <mark>
    This section is currently under draft on this [Google Doc](https://docs.google.com/PATH/TO/GOOGLE/DOC).
    We welcome help and feedback at all levels!
    If you would like to contribute, please see
    [Contributing to the Guidelines](https://github.com/transparentstats/guidelines/wiki/Contributing-to-the-Guidelines).
    </mark>
    
  2. Port to markdown/git. Once developed to a point where broader feedback is desired or once there are no more edits and comments for 5 days, FAQ is ported to Markdown in a separate branch or fork of the main github repo (see the Style Guide for how to name related FAQ and exemplar files). All work on that topic will be conducted on that branch (not master) until it has passed initial review. If all initial authors of the topic have access to the main repo, one suggested approach is to create a branch on the main repo named after the topic. Otherwise, it may be necessary to either request access to the main repo or create a fork on a separate repo and give all desired authors access there.

  3. Open pull request to track review. Authors create a pull request to merge the topic branch onto the master branch. The pull request should be titled First version of [topic] topic and have the review label, and should have two reviewers with subject matter expertise who were not involved in the creation of the topic assigned to it. This checklist should be the contents of the first comment in the pull request:

    - [ ] First review by another community member *with* subject matter expertise
    - [ ] Second review by another community member *with* subject matter expertise
    - [ ] Check & fix coding style
    - [ ] First test by another community member *without* subject matter expertise
    - [ ] Second test by another community member *without* subject matter expertise
    
  4. Update pull request as review proceeds. As the above checklist is completed, the initial comment should be edited to tick those boxes like so:

    - [x] First review by another community member *with* subject matter expertise
    - [ ] Second review by another community member *with* subject matter expertise
    - [ ] Check & fix coding style
    - [ ] First test by another community member *without* subject matter expertise
    - [ ] Second test by another community member *without* subject matter expertise
    

    Coding style: see Style-Guide

    All comments and discussion about the topic should be made on this pull request so that everyone involved in the review is notified and so that relevant comments and changes are documented.

  5. Accept request to merge topic onto master. Once the above reviews are complete and all boxes checked, the pull request is accepted by a guidelines-core member who is not one of the primary topic authors.

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.