Handling flood of trivial PRs

[nikhita]

If you are a new contributor, please note that all PRs are welcome in @kubernetes - even if it is a small typo fix! This issue is for discussing how to handle a flood of trivial PRs for reviewers, approvers and members.

---

We have recently noticed a flood of trivial (typos, removal of repeated words, punctuation) PRs in many repos. This situation has been discussed in the #sig-contribex channel on Slack and this issue captures some of the discussion.

Current Situation:

What the problem is NOT about: New/new-ish contributors submitting PRs with trivial changes. It is absolutely fine to start with something small so that they can learn the process.

What the problem is about:

• Trivial changes are being sent as separate PRs by the same author instead of being batched into a single PR.
• There is at least one new PR each day with a trivial fix.
• Some authors are only creating trivial PRs across several repos in the @kubernetes org.
• If a reviewer suggested changes to a (trivial) PR, the author simply files a new PR and tries to assign to someone else, sometimes even to people who are not in the relevant OWNERS files.

Problems to reviewers:

• Reviewers feel overwhelmed at the sheer volume of such PRs.
• It affects reviewer bandwidth and makes it harder for them to tackle non-trivial PRs.
• Leads to lots of noise to the PR queue.
• Creating new PRs while disregarding the changes requested on the old PR leads to frustration amongst reviewers.

Possible motivations for creating such PRs:

• To game the number of contributions in devstats and stackanalytics. Gaming the numbers in devstats could lead to a higher individual/company ranking.
• We noticed a sudden spike in such PRs when it was announced that eligible voters for the Steering Committee elections require at least 50 contributions in devstats. It is possible that people wanted to qualify as a voter and started creating trivial PRs.
• To gain org membership by having more number of PRs under their belt.

Possible Solutions:

We already have documentation for authors submitting trivial changes. Some more solutions for reviewers discussed on Slack:

• /holding PRs which tackle only few (one or two) trivial changes so that the author can add some more changes to the PR before merging.
  • Suggestion on how to do this (can be with/without holding): "Thank you, we really appreciate your contribution. It looks like you're great at this, would you mind going over the entire document and see if there are other things that needs fixing as well?"
• Point authors to good first issues by explicitly (and politely) asking authors if they would like to tackle them.
  • Suggestion on how to do this: "I've noticed you've submitted a few of these and probably now understand how the process works. Would you mind looking over this policy here and start working on larger commits? We like to see people grow as they contribute and we think you're ready to take on more complex tasks".

Automation:

• While we do have automation in place for catching typos, the library used for spell-check (misspell) is lacking and does not catch lots of mistakes.
• We don't have automation in place for catching repeated word typos. It is somewhat possible manually by Remove duplication #2898 (comment) but the approach is not automation-friendly.

xref https://groups.google.com/d/topic/kubernetes-sig-docs/aapjkJ8gD1c/discussion - This problem was also discussed on the sig-docs mailing list last year.

/sig contributor-experience

[duglin]

One option is to allow someone with the proper authority to tag a PR with a certain tag that would skip the testing phase when the PR does things like fixes a typo in comments. While its a bit frustrating to see a flurry of those PRs, I'm more annoyed at knowing that its slowing meatier PRs from getting access to the testing resources.

[castrojo]

Perhaps we can rate limit submissions? For example if one author who isn't in the org files more than 1 XS issue in a span of X hours the bot removes the PRs from the usual queues by unassigning the usual tags and then pastes in a link to the trivial edit policy. Then perhaps a bunch of us work through a "new" queue so that we can mentor them through the initial process?

[nikhita]

We briefly discussed this on the November 21st SIG Contributor Experience meeting. The consensus was that we probably need metrics on the volume of trivial PRs and how much of an impact/problem it is for the community before moving ahead with any policy and/or automation changes. The current feedback about this problem is pretty anecdotal.

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[nikhita]

We still get a few trivial PRs in k/k weekly but I don't think there is sufficient data that it is immensely problematic today.

I'll close this issue since it isn't really actionable without metrics of negative impact (if any) trivial PRs have had. Please reopen if needed, or create a follow up issue discussing the metrics aspect.

/close

[k8s-ci-robot]

@nikhita: Closing this issue.

In response to this:

We still get a few trivial PRs in k/k weekly but I don't think there is sufficient data that it is immensely problematic today.

I'll close this issue since it isn't really actionable without metrics of negative impact (if any) trivial PRs have had. Please reopen if needed, or create a follow up issue discussing the metrics aspect.

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.