MSC4073: Shepherd teams

[joepie91]

An attempt at addressing the problems with SCT throughput.

Rendered

As usual, please make sure to attach any review comments to a changeset line!

[turt2live]

@joepie91 when you get a chance, please sign off on the changes for this MSC to be eligible for FCP acceptance. Also, lines should be limited to approximately 120 characters.

[turt2live]

Process-wise, I think it's actually best if we sit on this and think about it for a while (ironically). The MSC is largely written in response to recent events, and we should be cognisant that those events may be clouding judgement. Similarly, we should not rush into decisions when it comes to process: they need careful thought and consideration, even if appearing simple on the surface.

[joepie91]

The MSC is largely written in response to recent events

That is not quite accurate; none of these issues are new, and all of them have been brought up in a more informal manner many times, including suggestions to have the SCT do less. What has changed recently, are two things:

1. The Foundation is now showing itself more receptive towards governance changes than before, and
2. The situation has worsened to a degree that the Foundation itself expresses public concern about a lack of third-party contributions, and there is a lot of unrest in the broader Matrix community about what this means for the future of Matrix, for which there need to be concrete answers.

So rather than this being a sudden novel idea, it's an idea that has been floating around in various form for quite some time already, and has already proven its worth in another community (which has actually partly modeled its spec process on that of Matrix, to my knowledge), and it is now at a point where concrete solutions need to start appearing, so as to not to lose the remaining trust of the community.

Similarly, we should not rush into decisions when it comes to process: they need careful thought and consideration, even if appearing simple on the surface.

Of course, careful consideration is important, that is part of why I submitted this as an MSC rather than trying to backchannel it. But this does not need to translate into a long approval process in terms of time, and it's not clear to me in what way this proposal would be expected improve by letting it sit, in and of itself.

In this consideration, given the circumstances and the worsening community trust issues, I also think it is important to ask: what possible worst-case scenarios are there for the outcome of this proposal, that are worse than the outcome of not changing the process?

Because governance change proposals do not need to be perfect to be useful, they just need to be an improvement over what we currently have, so if it turns out that something was overlooked, this can always be rectified down the line (especially since the SCT still retains the power to intervene if really necessary).

[joepie91]

please sign off on the changes for this MSC to be eligible for FCP acceptance

Completed privately.

Also, lines should be limited to approximately 120 characters.

Fixed.

[InezMc]

Private signoff has been recieved.

[clokep]

Due to the use of GitHub the shepherds probably would need to depend on the SCT to resolve comment threads and such. This is a bit annoying, but hopefully not a huge deal.

They might also need to rely on them to modify the text (via merging PRs?) or getting access to the original source repository to modify the pull request.

[joepie91]

Due to the use of GitHub the shepherds probably would need to depend on the SCT to resolve comment threads and such. This is a bit annoying, but hopefully not a huge deal.

I assume this comment is under the assumption of the second option in your other comment? If so, it would not be relevant in this case, as the author does principally retain control over the MSC, and the author can also resolve conversations themselves (so SCT intervention would only be necessary in the case of a dispute between author and shepherd team).

They might also need to rely on them to modify the text (via merging PRs?) or getting access to the original source repository to modify the pull request.

Good point, this will require some sort of solution. Given that "editing responsibility is transferred to shepherd team" should be the exception rather than the rule, I feel like it's acceptable for this to require some setup - even if a contributor does not wish to grant the shepherd team access to their fork as a whole (I don't remember to what degree this can be branch-restricted), in the worst case, I imagine a dedicated fork with shared access could be set up just for the purpose of shepherding that one specific MSC.

I'm not sure whether it's possible to repoint an existing MSC PR to source itself from a different fork after-the-fact, though; if not, that may be a problem for this approach.

Do you feel that the exact procedure for such a responsibility transfer should be included in this MSC? Or would it be acceptable to leave it as an unspecified implementation detail, and/or only address it in a separate MSC after this one?

[uhoreg]

It might be possible to just add shepherds as maintainers to this repository, and make sure that MSCs have the "Allow edits and access to secrets by maintainers " setting enabled. Presumably, shepherds should be able to be trusted to behave properly and not abuse their privileges (e.g. by interfering with other MSCs).

[joepie91]

That's a point I hadn't considered yet. I think that's a viable option, but it's also going to depend on whether all the current SCT members feel comfortable with that approach.

[clokep]

I assume this comment is under the assumption of the second option in #4073 (comment)?

Only somewhat -- if the author is unresponsive or if the shepherds are deciding what to do with an MSC, they seem like they would need some editorial power to resolve threads, etc.

Just to be upfront -- I don't think the technology not being perfect here is something that should block this proposal, just wanted to point out a particular sticking point.

[joepie91]

Only somewhat -- if the author is unresponsive or if the shepherds are deciding what to do with an MSC, they seem like they would need some editorial power to resolve threads, etc.

Right. I'd be inclined to classify this as an exceptional case requiring administrative intervention; if the author is unresponsive, and no explicit transfer of editing responsibility has taken place, then (assuming shepherds do not straight-up get access to the spec repository) it would be up to the shepherd team to request administrative intervention from the SCT.

As long as this doesn't happen frequently, it sounds sufficiently workable to me as a solution. And if it does happen frequently, and authors frequently abandon MSCs, then that is IMO a bug in the specification process that needs to be addressed - because we shouldn't be seeing that sort of abandonment rate in a healthy spec process.

[ShadowJonathan]

Speaking from my experience of being a shepherd (on #2666), and from a past idea of mine; This problem can be fixed by introducing a bot that can answer to commands like /res, /edit, or other such functions.

This may increase noise, but it would at least allow the shepherd some control over otherwise locked functions of a shepherded PR.

One issue I also had was the inability to push commits to the PR branch; this can be solved by instructing the bot to push commits onto the PR branch, targeting some branch the user is currently working on. (Force pushes would be disallowed, and only fast-forward merges/tracking would work)

[uhoreg]

There are some cases where an MSC is fairly trivial, and everyone just agrees that it should be done (MSC3828 might be a good example of this). For such MSCs, creating a shepherd team for it might be unnecessary administrative overhead, and I wonder if it might be worthwhile to keep a "shortcut" of having the SCT accept the MSC. I'm not sure if it happens often enough, though.

[joepie91]

That is a fair point; I'm not opposed to adding such a shortcut, but I do think that it should be a well-defined one, such that that path is only taken when there's reasonable certainty that it can be passed quickly. My main concern is that if there isn't such a defined threshold, it can become appealing for SCT members to take on more work than they can realistically handle, out of excitement/optimism.

I'm thinking something along the lines of:

If the SCT believes that the proposal can be passed within 3 weeks, then the SCT itself may function as the shepherd team for that MSC, skipping the shepherd assignment process. If, against expectations, after 3 weeks the proposal has not yet been merged, the MSC is no longer eligible for this accelerated treatment, and the normal shepherd assignment process must be followed belatedly.

... but I'm open to other suggestions as well.

[uhoreg]

Personally, I'm not a fan of making too many rules unless necessary. If we do want to add this shortcut, I'd be more in favour of either making it a guideline rather than a rule ("if the SCT hasn't managed to get their act together within 3 weeks, then they should really consider assigning it to a shepherd team"), or leaving it open and then fixing any bugs if they occur later on.

[uhoreg]

Thanks for writing this up. In general, I think this is an idea worth exploring.

[uhoreg]

Obviously, this will depend on peoples' willingness to be shepherds, so we'll need to see how much involvement we can get. I suspect that the community is at a point where we should be able to get shepherds in at least most subject areas. Some other areas, I'm not so sure about. We may need to find out how many people there are who would be willing to be shepherds, in what areas, and how much time they can commit to.

[joepie91]

I suspect that initially, it will be difficult to find enough shepherds, because a lot of people have already (halfway) given up on the spec process due to past experiences, and it will take time to 'rebuild trust' on that point.

Eventually, I expect this problem to resolve itself; it is already common for implementers to involve small communities of people for review, and for people outside of that community to have critical feedback in more informal ways. I think that a lot of that feedback and motivation can be moved into shepherd roles in the long term.

Given that I expect a somewhat rocky start, it may be a good idea to deal with the existing MSCs by slowly and gradually moving them over to the shepherd model, to get them out of the weeds. There are a number of MSCs I can think of that a lot of people are passionate about, and that have also raised criticisms, and which would probably be good candidates to try out the shepherd process on to start with. If this goes well, that can be expanded to other existing MSCs.

How to deal with existing MSCs should probably be part of the proposal too... I will add that soon as well.

[joepie91]

Relatedly: clear guidance will be necessary for shepherds, on how to conduct their part of the job, so that they can work independently without burning out. This is more of a note-to-self, in that I should add this to the proposal.

[joepie91]

(Both of those points have now been added to the proposal.)

[uhoreg]

Another minor point: some times, multiple conflicting MSCs are submitted that solve a problem in different ways. I think it makes sense to have the same shepherd team to those MSCs to sort out which approach to take. In this special case, excluding the authors may unnecessarily limit the pool of shepherds (as, instead of just excluding one person, we may now be excluding, say, three people), and it may even be beneficial in such cases to have the authors of conflicting MSCs co-shepherd the proposals. I'm not sure what would be the best way to handle this exception.

[joepie91]

Thinking through it a bit, I'm not certain that that approach would be ideal. The final state of an MSC does not necessarily look much like its initial draft, depending on how much refinement and feedback it has gone through, and especially for more complex design problems, it is very possible that what initially seemed to be the worse solution, actually ends up being the optimal one after a few rounds of refinement.

Given that, I think it would actually be beneficial for conflicting proposals to proceed independently, with the understanding that coordination between the shepherd teams is needed before any FCP is called; so that each proposal can independently develop, and the comparison ends up being between the refined proposals, rather than between drafts. In the current model, this would place a significant burden on the SCT, but in a shepherd-driven model, that would not be a concern; they would be separate teams, like with any other two unrelated MSCs.

Eventually, the author may decide to withdraw their proposal because they feel that the alternative is better (in which case the relevant shepherd team is disbanded), or when both proposals have reached a near-FCP stage, the shepherd teams might in their coordination decide that only one of them is worth moving ahead with, or even decide that a merge between the two should take place (in which case the SCT could reorganize the shepherd team with an exemption from the usual no-authors rule). That would be dependent on the specific case.

I feel like this is more a matter of "providing good guidance to shepherd teams, for coordinating this stuff", than a matter of having a strict written no-conflict policy.

[uhoreg]

Yeah, it may be better for conflicting MSCs to proceed independently. I think we should just leave this open as something to consider.

[joepie91]

Note: I have belatedly added this section to explicitly call out that shepherds are responsible for seeking review from existing implementers; I believe that this is currently a task that the SCT does, but it requires insight into the details of pending proposals that the SCT may no longer (always) have under this proposal.