Join GitHub today
GitHub is home to over 28 million developers working together to host and review code, manage projects, and build software together.Sign up
Proposal: use tooling that encourages reviewers #22120
First off, I'd like to start by saying I appreciate the work the Go team does. Working with the community while developing a programming language, compiler, and toolchain is no small task. And, everyone has opinions which can be difficult to navigate.
I can understand how scaling qualified reviewers can be difficult. Many open source projects deal with the same problem. All to often they write their own custom tools to support them along the way. I currently work on Kubernetes which is struggling with scaling out people while writing custom tools to work around GitHub shortcomings.
But, I can appreciate how tool choice can have an impact on reviewers. In a post on lwn, Jonathan Corbet noted:
Having used Gerrit quite a bit on OpenStack I appreciate this problem for reviewers.
Back in 2015 I started to investigate bettering the Gerrit usage situation. For example, here's a twitter conversation with Emma Jane (author of the book Git for Teams). I note this conversation because it's public.
For most developers and reviewers, Gerrit is not an ideal tool. It's not a tool of choice or one they enjoy using. As was noted...
This may not be a universal truth but it appears to be the rule with some who are exceptions to it.
This proposal is open ended at the moment without a suggestion for a tool because it's worth knowing if there is acceptance of going down this path prior to putting in the effort to do so. If the Go team is not interested than the proposal need not progress further. If the Go team is honestly interested then criteria, user stories, and other details can be collected along with tools and processes being evaluated against them.
Is there interest in tooling to increase the number of reviewers?
I would also like to see better review tooling. But Gerrit, at least as used by the Go team, is better than what you are describing. You quote Jonathan Corbet quoting an unnamed person:
The Go team has scripted Gerrit using the
While Gerrit does show each file separately, you can move between files easily using the
I'm not sure what local testing of patches means, but the Go trybots are fast and effective.
Tracking a patch series is the opposite of impossible: it is easy. See "Multiple-Commit Work Branches" at https://godoc.org/golang.org/x/review/git-codereview.
While it is true that most discussion in Gerrit are through a web interface, it does in fact support e-mail.
I can not disagree more. I've done reviews in e-mail and Github, and I strongly prefer Gerrit.
@ianlancetaylor I'd like to provide a few clarifications...
I'm sorry I didn't make it more clear. At the start of the linked article he notes the person is Greg Kroah-Hartman who, among other things, is the "the current Linux kernel maintainer for the
This is my proposal title on purpose. I didn't suggest moving away from Gerrit. And, you note that there are some tools on top of Gerrit that the Go team has built to help improve the process. That's great and a start.
But, I will posit that the current setup doesn't encourage new reviewers who aren't being paid to do it. Unless you are not looking for more reviewers, wouldn't it be a good idea to look at tooling that encourages reviewers? Including new ones.
When I used Gerrit I became comfortable with it. Like Go, the OpenStack team also had tooling to make working with Gerrit easier which I appreciated.
But, you and I are not typical people. We are both in the "tribe" when it comes to using Gerrit. As Emma Jane noted, "people haven’t begged for info about it." In fact, finding good details on navigating Gerrit is hard. Even the contribution guide is focused on contributing and not reviewing. The contribution guide is long (rather than intuitive to new folks).
To get into reviewing you need to figure things out, much of which is not documented well and is not intuitive, or more likely know people in the "tribe" who can teach you.
Do you want to "use tooling that encourages reviewers" or to use tooling that the current reviewer team is comfortable with? Even if the current tooling is a turn off is getting new reviewers.
Please note, I'm not talking about what I prefer or would prefer. I'm approaching this as a people problem and figuring out how to encourage more reviewers and lowering that bar to entry and repeat performance with a bit of joy.
I think the barrier to entry for reviewers isn't the tool they use. It's the need to understand the codebase that the change is modifying. Making changes to the review tool for reviewers, in order to bring in more reviewers, seems like just tinkering around the edges.
Just about every Go reviewer started out as a contributor. I think the contributor experience is where any effort should be spent making the process better (e.g. issue #18517). Having more contributors is by far the best way to have lots of people understand the codebase, which is the best way to expand the reviewer pool.
I don't believe that the 'outsiders' you'll (maybe) please by switching away from gerrit will be able to contribute to the review process in a meaningful way .
You can (roughly) partition present or potential contributors in three groups:
The people in the first group do 90% of the review work. This is expected, because they are the 'code owners', the people that have the last word about a proposed change, and the most knowledge. Those people seem to like gerrit.
The people in the second group are the ones you should try to please if you want to increase the number of reviewers. I rarely see people in this group help with reviews. There are (almost) no 'code-owners' there, but they can still make meaningful contributions to the review process. I suspect that most people in this group actually like gerrit. Well, we could ask them. I'm in this group, and I like gerrit.
People in the third group are the ones that we can hope to board as contributors (i.e. people who report bugs and send patches), but they do not have the knowledge necessary to contribute to the review process, and I believe it would be a mistake to try to cater to them. And when someone from this group moves to the second, it's more than likely that she'll understand gerrit enough to do reviews.
I do work for Google (now), but my background is that I started contributing to Go as just some random contributor and I favor Gerrit. I still have to use GitHub PR's on a regular basis, and it makes me sad. Sure, I wasn't begging to use Gerrit, but when I starting using it more, I saw how it was a superior tool to GitHub.
As somebody who's contributed a handful of times, I can't even say I know the proper process to help review code. I think the docs for the entire lifetime of a contribution to Go could use some TLC to make them simpler.
The contribution guidelines say that it does not. Are they simply out of date or are you referring to something different than the guidelines? Specifically it says:
(Regardless the mention of Rietveld seems past its expiration date at this point).
I've heard enough people whose opinions that I trust say that Gerrit is superior to be willing to believe it, though I am not yet familiar enough with it to form that opinion for myself.
Even if optimizing for contributors is the goal, contributors—at least currently—need to use most of the same features of gerrit as reviewers so documenting common tasks and workflows and increasing automation seem like good ideas regardless.
I'm also not sure that more casual reviewers would be a bad thing. If someone comes along before the code owner and points out some typos or possible simplifications to the code, etc., then, sure, there's still all the important stuff to work out but all the misc. simple stuff has been cleared out of the way so that can be focused on. Personally, I lurked in reviews for quite a while and pointed out some incorrect docs on a CL before even attempting to contribute to get an idea of the process and how to use gerrit at just the most basic level. Basic gardening in code reviews can be a gateway to contribution.
The contribution guidelines are out of date. You reply to emails sent by Gerrit and the response is reflected in the review. More info here: https://gerrit-review.googlesource.com/Documentation/intro-user.html#reply-by-email
...or you can click the triangle to expand them individually.
In fact, I would encourage everyone to look around in settings to see if there’s anything that would make your workflow a little better. There are a few good ones in there.
Both the Twitter conversation and post on lwn predate the broader launch of the PolyGerrit UI that the Go team now uses by default. This project has been a multi-year effort to address the usability problems and feature desires of the many developers using Gerrit and will usurp the old UI once it reaches feature parity.
This is all being done in the open (bugs and source). So while I completely agree that Go could be doing a better job of making things more approachable and clear, deficiencies with Gerrit itself should be reported to them as they are much better equipped to handle those.
pushed a commit
Oct 4, 2017
A few quick comments...
Instead of debating between GitHub and Gerrit (a common conversation which I'm guilty of falling into the trap of sometimes), is there interest is looking at toolchains to improve the reviewer experience with a goal of bringing in more reviewers?
@andybons The impressions that I get from this thread is "you can drag my current workflow from my cold dead hands" and "Gerrit is better than X". I'm left with the feeling that suggests are welcome and will them be met with same attitude. Why will someone feel it's really worth their time?
Because cogent, detailed proposals that outline plans to encourage more reviewers are a good thing.
From reading the above thread again, it doesn't seem like there are any concrete proposals on how you wish to encourage more reviewers; only that you would like to investigate possible solutions.
Yes. Absolutely feel free to investigate and present possible solutions to the problems you outline. We will treat them as we would any other proposal, so I encourage you to go for it.
@mattfarina I'm sorry if we seem to be rebuffing your suggestion. I don't see anybody on this thread saying that Gerrit is perfect. It's not. It's a very long way from perfect. We're still recovering from #21956 and that is probably coloring our responses somewhat.
I guess I don't know how to evaluate your proposal. Everything is a trade off. Switching away from Gerrit carries some real costs. Whatever we switch to would have to be clearly and significantly better. You seem to be asking whether the Go team is interested in switching. My answer: the Go team would love to switch to something that is clearly and significantly better. The Go team is not interested in switching to something that approximately the same.
It seems to me that the simpler approach is improving Gerrit. Gerrit has completely revamped their UI, and in general they seem to be open to bug reports and suggestions. But I'm not wedded to Gerrit. I just think that there is a reasonably high bar to changing.
You are positioning this as tooling that will bring in more reviewers. Like @randall77, I don't think the tooling is the barrier. Of course we should make the tooling better if we can. But I don't see that as likely to change the set of reviewers one way or another.
The proposal process is for concrete, specific suggestions that we can evaluate. It sounds like you want to have a discussion. One of the mailing lists would be a better forum for a discussion.
As Ian said, the Go team is not interested in switching to something that is approximately the same, so I am honestly skeptical about the framing here. You'd do better by focusing on what pain points are and how we might get them improved. There's a TON about Gerrit that works very very very well for us, and it's doubtful any other systems are as well molded to our processes at this point.
But again, this is not a proposal, so closing.