Skip to content
This repository has been archived by the owner on Dec 15, 2022. It is now read-only.

RFC for Pull Request Review #1706

Merged
merged 39 commits into from Oct 4, 2018
Merged
Changes from 13 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
43b33e3
Screw it, press commit
smashwilson Sep 27, 2018
6af2960
and not or
smashwilson Sep 27, 2018
48a31dc
A little more justification
smashwilson Sep 28, 2018
438a9e6
Prose for the rest of the UI components
smashwilson Sep 28, 2018
bcc742d
Drawbacks and rationale
smashwilson Sep 28, 2018
e5410a3
Review creation
smashwilson Sep 28, 2018
3b7ac4a
Use real headers
smashwilson Sep 28, 2018
85ba758
Kind of answered that one myself
smashwilson Sep 28, 2018
194d362
Add a mock-up for non-current pull request tiles
smashwilson Sep 28, 2018
81a53e0
Update review-list mockup
simurai Oct 1, 2018
12532be
Add reviews-tab mockup
simurai Oct 1, 2018
3279a26
Update pending-review mockup
simurai Oct 1, 2018
d902dcf
Replace changes and reviews tab mockups
simurai Oct 1, 2018
64f2433
"position: sticky" navigation and filter banner in "Changes" tab
smashwilson Oct 2, 2018
0cb878e
Unify the "Reviews" and "Changes" tabs
smashwilson Oct 2, 2018
e50fa72
Emphasize review authoring and finalization on the "Changes" tab
smashwilson Oct 2, 2018
884c39d
Mention the review summary list
smashwilson Oct 2, 2018
c37fe1f
Reaction emoji ftw :cake: :tada:
smashwilson Oct 2, 2018
c45f9de
Explicit "single one-off comment" handling
smashwilson Oct 2, 2018
792d9d0
Add answers to some of our questions :smile:
smashwilson Oct 2, 2018
5465f71
Oh hey that works
smashwilson Oct 2, 2018
ea33834
Dependency graph why not
smashwilson Oct 2, 2018
d4b153b
Update iconography description
smashwilson Oct 2, 2018
6bccb4b
Suggestion for reducing confusion when a different PR is open
smashwilson Oct 2, 2018
759884a
Remove non-current pull request tile changes
smashwilson Oct 2, 2018
af4c794
IssueishDetailItem not IssueishPaneItem
smashwilson Oct 2, 2018
6313744
Include @kuychaco's IssueishDetailItem refocus idea :lightbulb:
smashwilson Oct 2, 2018
2ab74b5
Create a pending review if one is not present
smashwilson Oct 2, 2018
a7b7bae
Update RFC to reflect editor-first design
kuychaco Oct 3, 2018
cd41db4
Incorporate feedback
kuychaco Oct 4, 2018
29c6a6e
Add in-line comments back
kuychaco Oct 4, 2018
58863e1
Merge pull request #1712 from atom/ku/pr-review-rfc-editor-focus
smashwilson Oct 4, 2018
ea3973b
Mark remaining TODOs as :construction:
smashwilson Oct 4, 2018
2750281
Add an Explanation section for in-editor comment rendering
smashwilson Oct 4, 2018
c445668
Move diff editability questions to "answered during implementation"
smashwilson Oct 4, 2018
17ec937
Move this question from "before merge" to "during implementation"
smashwilson Oct 4, 2018
e6c34ab
Dependency graph bump
smashwilson Oct 4, 2018
bad5770
Give it a number :tada:
smashwilson Oct 4, 2018
5ad24e6
Status: accepted :hammer:
smashwilson Oct 4, 2018
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
145 changes: 145 additions & 0 deletions docs/rfcs/XXX-pull-request-review.md
@@ -0,0 +1,145 @@
# Feature title

Pull Request Review

## Status

Proposed

## Summary

Give and receive code reviews on pull requests within Atom.

## Motivation

Workflows around pull request reviews involve many trips between your editor and your browser. If you check out a pull request locally to test it and want to leave comments, you need to map the issues that you've found in your working copy back to lines on the diff to comment appropriately. Similarly, when you're given a review, you have to mentally correlate review comments on the diff on GitHub with the corresponding lines in your local working copy, then map _back_ to diff lines to respond once you've established context. By revealing review comments as decorations directly within the editor, we can eliminate all of these round-trips and streamline the review process for all involved.

Peer review is also a critical part of the path to acceptance for pull requests in many common workflows. By surfacing progress through code review, we provide context on the progress of each unit of work alongside existing indicators like commit status.

## Explanation

### Current pull request tile

Reviews on the current pull request are rendered as a list on the current pull request tile.

![review-list](https://user-images.githubusercontent.com/378023/46273505-b9533280-c590-11e8-840e-a8eac8023cad.png)

* The review summary bubble is elided after the first sentence or N characters if necessary.
* Clicking the review summary bubble opens an `IssueishPaneItem` in the workspace center, open to the reviews tab.
* Clicking a line comment opens or activates an editor on the referenced file and scrolls to center the comment's line, translated according to local changes if appropriate.
* Line comments within the review are rendered: _with a dot_ before the file has been opened and the corresponding decoration is visible; _with no icon_ after the file and decoration have been seen; and _with a checkmark_ after the comment has been marked "resolved" with the control on its decoration.

If a new review has been started locally, it appears at the top of the "Reviews" section within this tile:
smashwilson marked this conversation as resolved.
Show resolved Hide resolved

![pending-review](https://user-images.githubusercontent.com/378023/46275946-9bd69680-c599-11e8-9889-66c35458286a.png)

* The review summary is a TextEditor that may be used to compose a summary comment.
* Choosing "Cancel" dismisses the review and any comments made. If there are local review comments that will be lost, a confirmation prompt is shown first.
* Choosing "Submit review" submits the drafted review to GitHub.

### Non-current pull request tiles

Pull request tiles other than the current pull request display a one-line review summary, showing the number of accepting, comment, and change-request reviews made on each. Clicking the review summary opens the `IssueishPaneItem` for that pull request and opens the reviews tab.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the main purpose to have a quick way to navigate to the "Reviews" tab? Or to keep an eye on new reviews? For the later, some sort of "new comments since I last visited", might be handy, but comes with its own challenges. It also looks a bit crowded with all the icons and numbers.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the main purpose to have a quick way to navigate to the "Reviews" tab? Or to keep an eye on new reviews?

Yes 馃槅 It's also to start using the PR result tiles to show a bit more information without having to click through, and to give a little symmetry to the information you see for current and non-current PRs.

I'm not 100% sold on it myself though mostly because:

It also looks a bit crowded with all the icons and numbers.

So if it's just too messy we can yank it.

Sidenote:

For the later, some sort of "new comments since I last visited", might be handy, but comes with its own challenges.

I'm deliberately punting anything related to notifications about new information arriving from this RFC. It's still important and a place we can add a ton of value, but I believe it's something we should address in a consistent way across the package as a whole (controls that let you opt-in to Atom notifications? Blue-bars to signal "there's something new here"?). It's also something that's going to take a lot of us getting it wrong before we figure out how to get it right.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's also to start using the PR result tiles to show a bit more information without having to click through

Maybe we could show the "reviews" progress bar?

image

And maaaaybeeee also a "tasks" bar.

non-current pull request tile

Then you can glance over the PRs and get a sense of their progress. Like " 馃槺 Oh, this looks almost ready, I better add my review."

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I definitely like yours better than my mockup, but I'm still not convinced we shouldn't just leave non-current PR tiles alone. I'm kind of leaning toward not touching it I guess?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clicking the review summary opens the IssueishPaneItem for that pull request and opens the reviews tab.

Something to consider: prompting the user to see if they want to check out the corresponding branch associated with the PR. Prompts can be annoying and sometimes feel intrusive. That said, if the user wants to make use of our fancy inline comments, they'll want to make sure that they are on the correct branch.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe when you're on a non-current PR, we could show a banner somewhere in the IssueishPaneItem tab that tells you to check out the PR to see inline review comments? Less intrusive than a modal but still a nudge that you're missing the full experience.


![non-current pull request tile](https://user-images.githubusercontent.com/17565/46228625-a2aea080-c330-11e8-945b-72be7824623f.png)

### IssueishPaneItem "Changes" tab

Each `IssueishPaneItem` opened on a pull request has a "Changes" tab that shows the full PR diff, annotated with comments from all reviews.

![changes-tab](https://user-images.githubusercontent.com/378023/46287431-6e9bdf80-c5bd-11e8-99eb-f3f81ba64e81.png)

* The up and down arrow buttons quickly scroll to center the next or previous comment within this tab.
* Clicking the :hamburger: button navigates to the "Reviews" tab, expands the owning review, and scrolls to the same comment within that view.
* Clicking the "code" (`<>`) button opens the corresponding file in a TextEditor and scrolls to the review comment decoration there.
* Clicking within the "Reply..." text editor expands the editor to several lines and focuses it.
* Clicking "mark as resolved" marks the comment as resolved with on GitHub. If the "reply..." editor has non-whitespace content, it is submitted as a final comment first.
* The "comment" button is disabled unless the "reply" editor is expanded and has non-whitespace content.
* Clicking "comment" submits the response as a new stand-alone comment on that thread.
smashwilson marked this conversation as resolved.
Show resolved Hide resolved

Hovering in the diff's gutter reveals a `+` icon that allows users to begin creating a new review with the same UI as described in the "In-editor decorations" section.

### IssueishPaneItem "Reviews" tab
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this seems to be equivalent to what dotcom shows under the "Conversation" tab for a PR. In our IssuishPaneItem we have an "Overview" tab which is similar to the "Conversation" tab on dotcom (but currently not interactive, namely you can't add comments yet). In our "Overview" tab we currently have timeline events like commits, references, and PR comments (standalone comments that are not associated with a particular review). The contents of the PR comments could be relevant to review comments.

So I'm wondering what makes sense in terms of how to divvy up information between our "Overview" tab and our "Reviews" tab. Does it make sense to combine them into one tab like on dotcom? Users would find this familiar and we could keep all of the comments (PR and review comments) in a single tab. On the other hand, it might be nice to have a dedicated "Reviews" section, because there's already a lot of other stuff going on in the "Overview" tab.

It would be good to get clear on the boundary and relationship between these two tabs if we want to have them both. Without having a strong narrative here, I would opt for combining them like dotcom has under the "Conversation" tab.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The contents of the PR comments could be relevant to review comments.

That's an interesting question. Sometimes I wonder how it would feel if PR and Review comments are strictly separated?

  • PR comments: More organizational, things to do, build failing etc.
  • Review summary and review comments: Discussion only about the content, the changes.

The only problem, I think you mentioned earlier, there is no place to make review comments without choosing a line. Review summaries could be used, but don't have replies and I think get overridden, when making a new review.

Somehow I'm still intrigued to try the separation, maybe because reading reviews in the conversion AND changes feels like double work?


Additionally, each has a "Reviews" tab that shows all reviews associated with this pull request in an accordion-style list. Unexpanded, each review is shown as its full summary comment and chosen outcome (comment, approve, or request changes). Expanded, its associated review comments are listed as well on their proximate diffs.

![reviews-tab](https://user-images.githubusercontent.com/378023/46287432-6e9bdf80-c5bd-11e8-8290-50dd17a2c733.png)

* The "Mark as resolved" and "comment" buttons and the "reply" text areas match their behavior in the "Changes" tab.
* The up and down arrow buttons and :hamburger: button are not present here.
* The "code" (`<>`) button also behaves as it does on the "Changes" tab.

A local, pending review created by the user is also shown at the top of this list, with controls to edit its summary comment and choose its final state.

> TODO: sketch here

### In-editor decorations

When opening a TextEditor on a file that has been annotated with review comments on the current pull request, a block decoration is used to show the comment content at the corresponding position within the file content. Also, a gutter decoration is used to reveal lines that are included within the current pull requests' diff and may therefore include comments.

![in-editor](https://user-images.githubusercontent.com/378023/44790482-69bcc800-abda-11e8-8a0f-922c0942b8c6.png)

> TODO: add gutter decoration?

* The comment's position is calculated from the position acquired by the GitHub API response, modified based on the git diff of that file (following renames) between the owning review's attached commit and the current state of the working copy (including any local modifications). Once created, the associated marker will also track unsaved modifications to the file in real time.
smashwilson marked this conversation as resolved.
Show resolved Hide resolved
* The up and down arrow buttons navigate to the next and previous review comments within this review within their respective TextEditors.
* The "diff" button navigates to the "Reviews" tab of the corresponding pull request's `IssueishPaneItem`, expands the owning review, and scrolls to center the same comment within that view.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth nothing -- my initial assumption was that the "diff" button would navigate to the "Changes" tab, not the "Reviews" tab. That said, I don't necessarily think that it's better to navigate to the "Change" tab...

Some good question to determine what makes sense might be --
"What information does the user care most about when clicking this button?"
"What action is the user interested in making next?"


Hovering along the gutter within a pull request diff region reveals a `+` icon, which may be clicked to begin a new review:
smashwilson marked this conversation as resolved.
Show resolved Hide resolved

![plus-icon](https://user-images.githubusercontent.com/378023/40348708-6698b2ea-5ddf-11e8-8eaa-9d95bc483fb1.png)

Clicking the `+` reveals a new comment box, which may be used to submit a single comment or begin a multi-comment review:

![single-review](https://user-images.githubusercontent.com/378023/40351475-78a527c2-5de7-11e8-8006-72d859514ecc.png)

## Drawbacks

This adds a substantial amount of complexity to the UI, which is only justified for users that use GitHub pull request reviews.

Showing all reviews in the current pull request tile can easily overwhelm the other pull request information included there. It also limits our ability to expand the information we provide there in the future (like associated issues, say).

Rendering pull request comments within TextEditors can be intrusive: if there are many, or if your reviewers are particularly verbose, they could easily crowd out the code that you're trying to write and obscure your context.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah this is a good point. It would be handy to offer a quick and easy way to toggle comments visible/hidden. Like a command/keyboard shortcut/easily accessible button.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I need some clarity on when these are being viewed. I would argue that you'd only want to see comments when you're in a review state, not in an editing state, making this a non-issue.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do have some state we can use if you're writing a pending review, but I don't think we have specific state right now that indicates whether or not you're actively reading reviews. Is there a natural place we can introduce that?


## Rationale and alternatives

One alternative may be to show review comments _only_ within the "changes" tab of an `IssueishPaneItem`. This simplifies flow considerably, because it removes the need to provide navigation among different views of the same review, and unifies the handling of current and non-current pull requests. However, I believe that the renderings of reviews in all three places each serve a unique purpose:

* Reviews within the "Changes" tab of the `IssueishPaneItem` reveal a narrative formed by all of the reviews on a specific pull request together, as a conversation among reviewers.
* Reviews within the "Review" tab of the `IssueishPaneItem` reveal the narrative flow within each review individually. For example, review comments that refer to other comments within the same review (e.g. "same here" or "and again") become clearer here.
* Review comments within open TextEditors allow the reader to use more context within the source code to evaluate, address, or respond to each individual comment thread: consistency with functions that are not visible within the immediate diff, context within algorithms that span many lines. They also allow the receiver of a review to preserve the mental context of the review communication as they move back and forth between reading the content of a review and applying it to their source.
Copy link
Contributor

@kuychaco kuychaco Oct 2, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In a world of multi-file editable diffs, I wonder if we could combine some of these in a natural and fluid way...

Something in my gut tells me that the "Changes" tab and the "Reviews" tab is too much / redundant / confusing... And they seem like the belong in the dotcom paradigm, whereas in our editor we want to create an entirely new paradigm around the fact that we are in the context of the user's working directory. Hence the power and beauty of editable diffs...

Here's what I'm picturing.... the picture is incomplete, just ideating here.

We could have a single view that shows all changes as a conversation among reviewers, and this view can be "filtered" to just show the changes for a single reviewer. This satisfies much of bullet points 1 and 2 above. I'm imagining a summary header that is always visible (floats there while the user scrolls much like the file-navigator on dotcom) that has quick summaries of each review and allows you to filter based on that review with a quick click. This header would include the progress information.

This view could be a multi-file editable diff that allows users to quickly apply changes to their code without having to click a "code" (<>) button and hop to a different view. This satisfies much of bullet point 3. Users may still choose to jump to a code file to get more context (say they want to view a function definition that is located elsewhere in the file and can't be seen by expanding context lines), but in the majority of cases I think users would simply want to edit the diff and incorporate changes from the comments.

With a single multi-file editable diff view we could potentially cover the bulk of the three scenarios described in a single view. Minimizing the back-and-forth and (hopefully) reducing the complexity of our UX (but maybe increasing it in other ways 馃 ).

I think the main point I want to make is that it feels a bit funky to me to have dotcom-style diff views in our IssueishPaneItem. We are in an editor. Users will most likely want to edit their source. Maybe we consider a editor-first solution?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, wow. Holy shit! This is a super cool idea.

We could have a single view that shows all changes as a conversation among reviewers [..]

Were you picturing this as the way we do the "Changes" tab (called that or something else)? Or is this something we add inline to the "Overview" tab somehow so it's colocated with the rest of the conversation?

Any thoughts on how it's ordered - do we render the files in diff order and attach comments according to placement? Or do we try to preserve the chronology of reviews somehow?

What do we do when you're looking at this view on a non-current pull request? If we had first-class worktree support, I could see us silently creating an isolated worktree for the PR and making your changes there.

I can also see it being confusing that an editable PR diff has different behavior than an editable FilePatchItem diff:

row change kind PR diff FilePatchItem diff
unchanged edit line directly in underlying file (readonly)
addition edit line directly in underlying file edit patch to be applied to index
deletion (readonly) (readonly)

I'd been thinking about editable diffs in a FilePatchItem as a rough equivalent editing the diffs in git add -p or git add -i, so you could do things like "delete a .only() in a Mocha test before it gets committed, leaving your working copy file alone." But we could also change how we do that to be more consistent with this 馃


## Unresolved questions

### Questions I expect to address before this is merged

Can we access "draft" reviews from the GitHub API, to unify them between Atom and GitHub?

How do we represent the resolution of a comment thread? Where can we reveal this progress through each review, and of all required reviews?

Are there any design choices we can make to lessen the emotional weight of a "requests changes" review? Peer review has the most value when it discovers issues for the pull request author to address, but accepting criticism is a vulnerable moment.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • The icon could change from "error" to "warning".
  • Replace "Request changes" with "Recommend changes". It's less demanding, but still with a slight urge. In the end it's up to the PR author to follow the recommendation (or not).

image

image

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

馃憤 I like that 馃槃

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah!! Some friends of mine wrote a Chrome extension to change the "request changes" icon on dotcom to 馃檹. Changing the icon can help a lot here to improve the tone.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, I kind of want to use the 馃檹 emoji now...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think @simurai's icon choices above are great too! .As long as we're not using the red icon dotcom uses, which feels pretty heavy handed to me, I'm open to a wide range of options.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love this question. And I love these suggestions. This sort of discussion and exploration makes me so happy :)


Similarly, are there any ways we can encourage empathy within the review authoring process? Can we encourage reviewers to make positive comments or demonstrate humility and open-mindedness?
smashwilson marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe even just emoji reactions to lines of code


smashwilson marked this conversation as resolved.
Show resolved Hide resolved
### Questions I expect to resolve throughout the implementation process

Review comment positioning within live TextEditors will be a tricky problem to address satisfactorily. What are the edge cases we need to handle there?
smashwilson marked this conversation as resolved.
Show resolved Hide resolved

The GraphQL API paths we need to interact with all involve multiple levels of pagination: pull requests, pull request reviews, review comments. How do we handle these within Relay? Or do we interact directly with GraphQL requests?

How do we handle comment threads?

### Questions I consider out of scope of this RFC

smashwilson marked this conversation as resolved.
Show resolved Hide resolved
What other pull request information can we add to the GitHub pane item?

Are there other tabs that we need within the `IssueishPaneItem`?
smashwilson marked this conversation as resolved.
Show resolved Hide resolved

How can we notify users when new information, including reviews, is available, preferably without being intrusive or disruptive?
smashwilson marked this conversation as resolved.
Show resolved Hide resolved

## Implementation phases

<!--
- Can this functionality be introduced in multiple, distinct, self-contained pull requests?
- A specification for when the feature is considered "done."
-->