Use review assignments for BinderHub team? #227
Comments
|
I'm very happy about having an automatic assignment and experiment with such workflows! |
|
I would be happy to integrate this into our team practices in some capacity. I must admit that my first response was "at this point I don't feel qualified to review most PRs to BinderHub" but that is largely because much of the time I've spent on the project is not digging in to code but doing other kinds of things. Something I'm concerned about in this case is that it assigns technical work in an equal fashion, but it is hard to equally-distribute the non-technical work in the same fashion.. Other ideas for ways that we could try and encourage more PR engagement would be to publish some team graphs that show the work people have done (monthly? quarterly?) in reviewing PRs, making PRs, commenting in issues, etc. I've got most of that data already here: https://nbviewer.jupyter.org/github/choldgraf/jupyter-activity-snapshot/tree/master/reports/ just haven't tried to integrate it into official processes because I'm worried about messing up the incentives and also rewarding technical over non-technical work... |
|
@choldgraf wow these reports are amazing! You present this really beautifully!!! I'm very excited to share this to and talk with my colleague Sharan Foga. Excited about this I detoured and found this talk, which I better save for tomorrow as it is way past my bedtime now! I think that in this video Sharan is presenting some of her work regarding the Apache community, and I believe that a lot of work and consideration you have done aligns with her work @choldgraf! |
|
I do it less often, but I do try to review code outside my knowledge domain from time to time, in projects I'm curious to learn about or care for. I have often learned a lot, but it certainly takes time. Btw, I would be happy to act as a mentor with some voice/video chats to help someone learn about z2jh if they would be interested in contributing to z2jh more actively. Perhaps it could be could be generally useful for projects vitality to have have some mentor for projects to help onboard someone looking to contribute more regularly? I'm now asking myself the question, how can we help overcome the initial learning hurdle that often comes along with the wish to contribute to a project? It can often be overwhelming. Hmm. |
|
I'd think of being assigned a review not as a "you must press merge (eventually)" but as "you are making sure someone does (eventually) press merge". In another team I work with we have a role called "issue board master" which is rotated every two weeks. Their role is to make sure issues that come in are taken care of. Either by doing it themselves or finding someone who can do it. Introducing it lead to two positive effects: the knowledge of who knows about what was spread more and we wrote a bunch of short guides on how to do things because we realised they were repeat questions that other team members didn't know how to do. One of the main ways the community interacts with Binder is through the repos so I think trying out some new things to help us be good at reacting to this interaction would be good. It doesn't mean we shouldn't also see if we can extend this to the forum, chat and other places like that. Should we try this for a few weeks or not? I'd vote against making a new team for this rotation as it IMHO would perpetuate the "tech work" vs "other work'. In a classic team you frequently get "backend" vs "frontend" vs "UX" splits because people are "frontenders". Besides the team dynamics aspects of this happening I think a small team like ours needs people who work on everything that comes up no matter what it is. This means the individuals have to be willing to do this as well as there being the resources (docs, instructions, guides, tools, mentoring, other humans sharing their knowledge, etc) available for people to be able to do this. |
|
I would recommend that a daily "how can you help" be posted in Discourse. New Issues, New PRs, Open Issues (past 7 days), Open PRs (past 7 days). Encourage any person from the community to review or respond. |
|
The "how you can help" post would be instead of this or in addition? I like the idea and maybe it solves the same problem. Should we try this instead of assigning reviews? |
|
Instead of assigning reviews. We could use a tool like org_pulse.rb or something else to generate markdown -> post on Discourse. Perhaps using GitHub Actions. My experience with Python is assigning is fine but it doesn't seem to speed things up. Getting more triagers does. See this as a first step to getting more triagers. |
I think something like this would be really neat. The jupyter activity snapshot repo does have tables for recent issues / PRs / etc. Maybe we could rip out some of that code to generate simple markdown tables for Discourse (or use a tool such as org_pulse.rb as @willingc recommended). What do people think about having some simple plots for things like "who has been reviewing PRs" each month? E.g., something like:
(I removed the usernames from above) As I mentioned before, I think publishing plots like is fraught with problems, but perhaps there is a way we can use it to give credit where it's due, and incentivize people to engage in particular ways? |
|
+1 to Carol's last comment (Assigning reviewers is fine but doesn't speed
things up). The underlying challenge is that everyones availability (even
people who work mostly full time on Jupyter) fluctuates over time. A class
of ideas that has come up in discussions about this in JupyterLab is to
require everyone who submits a PR to review N (where N=1-2) other PRs
before theirs is merged. The challenge is that you don't want to put up
barriers for new contributors, but maybe there is a creative way to handle
this.
…On Mon, Nov 18, 2019 at 7:16 AM Carol Willing ***@***.***> wrote:
Instead of assigning reviews. We could use a tool like org_pulse.rb or
something else to generate markdown -> post on Discourse. Perhaps using
GitHub Actions.
My experience with Python is assigning is fine but it doesn't seem to
speed things up. Getting more triagers does. See this as a first step to
getting more triagers.
—
You are receiving this because you are on a team that was mentioned.
Reply to this email directly, view it on GitHub
<#227?email_source=notifications&email_token=AAAGXUH7LUVCKPO3JDBAQD3QUKWVFA5CNFSM4JOL6632YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEEKYV7I#issuecomment-555059965>,
or unsubscribe
<https://github.com/notifications/unsubscribe-auth/AAAGXUFEGGMZBFHIIOJZQU3QUKWVFANCNFSM4JOL663Q>
.
--
Brian E. Granger
Principal Technical Program Manager, AWS AI Platform (brgrange@amazon.com)
On Leave - Professor of Physics and Data Science, Cal Poly
@ellisonbg on GitHub
|
I'm not a fan of publishing the plots for several reasons: PRs are not the same size or technical content so one cannot assume that the number of reviews == time spent reviewing; making it competitive motivates some and demotivates others. I think it is perfectly lovely to thank folks who have been active or did something helpful in a note pinned in an appreciation channel. It does two things: express gratitude and gives contributors something to show future employers. |
|
@willingc - thanks for those comments, those mirror many of my own concerns.
I like this idea - I wonder if this is something we could put in the community forum? I'm worried that top-level channels are already too crowded though I think it is important to lower the barrier to sharing positive feedback with one another. As one example of why I think this would be helpful - when I made that little jupyterhub activity repository, one of my goals was to find people who weren't on the github team but who were still engaging quite a lot across the repositories. When you zoom out and analyze activity across all the repos, I found people (both team members and non-team members) who were either: 1. Engaging a good amount, but distributed across many repositories, or 2. Engaging in a repository that was less well-trafficked. In both cases, the effect was that the contributions from those folks was relatively un-noticed. I'm trying to think of ways that we can make that kind of work noticed and appreciated. |
|
I greatly appreciate working with you on the Jupyter ecosystem @betatim @choldgraf @willingc @ellisonbg. I feel a great sense of purpose working alongside you with Jupyter matters.
|
|
It sounded like there was some support for having a "activity round up post" on the forum. So I went ahead and made one https://discourse.jupyter.org/t/binder-jupyterhub-activity-round-up-week-1/2704 One thing I realised just as I clicked publish: it will create a new message in every issue and PR I linked to which will generate a notification to lots of people. Sorry about that. @Carreau does meeseeks have a way to ignore a certain post on discourse? |
What do people think about trying out https://github.blog/2019-11-13-universe-day-one/#review-assignment to assign people from @jupyterhub/binder-team in a round-robin fashion for PRs in repo2docker, binderhub, jupyter-server-proxy and maybe some additional repos that are needed to run BinderHub (or maybe we use @jupyterhub/jupyterhubteam for other repos?)?
A reason to do it would be to help with getting PRs noticed and reviewed with little-ish delay and without people having to watch each repo and receive all notifications related to the repo. To start with I'd suggest we let the round-robin assign reviews with the understanding that the person who is assigned is responsible for reviewing that PR. For me "responsible" includes organising to hand it over to someone else if you can't do it.
Maybe we should remove some people from the team as they've not been active in quite a while and it seems unlikely they'd show up to review things?
The text was updated successfully, but these errors were encountered: