Add draft DEP dissolving Django core.

[ubernostrum]

Discussion has been taking place in the django-core and dsf-members mailing lists; refer there for background.

[jarshwah]

Is there a reason for delaying the process for merger selection until after this DEP is confirmed? I would think it'd make sense to confirm all process changes now, unless we're concerned that bikeshedding smaller decisions could impact or delay the overall DEP?

I mention this because I was specifically keen to see term limits and selection processes around the merger role, to avoid ending up in the same position we're currently in where the number of Mergers forever increases.

[FlipperPA]

I agree with the goal of what you're saying, but I don't necessarily think term limits may be necessary. Does this mean we'd only allow someone to hold the position of Django Fellow for a fixed amount of time, for example? I'd prefer some form of quantifiable activity required to maintain the merger role other than passage of time, as keeping people who are actively involved in merging (and the institutional knowledge that goes with such activity!) seems valuable.

[ubernostrum]

I'm not sure I understand the question; there are no term limits for Mergers here.

[jarshwah]

Correct - there are not. But I was specifically looking for a process that would avoid the current situation that we're in where mergers are never retired. Here's a brief example of what I would like to see:

• The set of mergers are automatically retired after some period X (maybe in line with the technical board, where a merger role is granted for a major series release)

• A merger CAN become a merger in the next term period

• The number of mergers can not exceed some limit X (say, 5)

Alternatively, we could review the set of mergers after each point release. "Who is interested/not interested in serving again? If we need more numbers, who from the framework team that meets the technical requirements would be interested in serving?"

In that way, the merger team would not be unlike that of the technical board. How the elections/appointments happen I'm less concerned with, and I think that could be handled by the incoming framework team. But I think some limits should be set sooner rather than later.

[collinanderson]

I'm interested in something like that too: You're automatically out of a certain period of time. I think that would help prevent burnout, and it would help encourage more people to "try it out" for a period of time. That way you're not just "in for life". (Fellows are an exception - they're in as long as DSF is paying them)

[ubernostrum]

@jarshwah if you'd like to write up an alternative proposal, go for it -- I did this basically because "let's fix the project organization" has come up so many times on mailing lists but nobody had ever taken the step of proposing how to do it.

[jarshwah]

I'm glad you have too. I have no interest in putting together an alternative proposal, especially if it were to only differ by some small details. I'd be happy to send a PR to amend some detail if you were interested though.

[collinanderson]

Ok, I've put together at least a rough outline for a proposal here: #50

[jarshwah]

A major series spans 27 months, give or take, is this an appropriate amount of time as a default? I recognise that the process for selecting the technical board will become more complex so that 9 months (a minor series) may be too short.

[aaugustin]

It seems consistent with the historically low turnover of the technical board.

[nemesifier]

I can't find the discussion regarding this on the mailing list, could you link it please?
I think adding a summary of the reasons that lead to this effort would be beneficial (eg: what is not working in the current model and what we want to achieve with the switch?).

[claudep]

I am very opposed to such an open membership with voting rights. This is an open door to potential problems. Nowadays, I assume it should be rather easy to create fake accounts, so with a little scripting ability, it would be feasible to change voting results. Couldn't we use the DSF membership as a requirement (or at least vouching by a DSF member)?

[timgraham]

I think using the DSF membership makes sense. That would eliminate the preregistration step in each election.

[jarshwah]

I don't think using the DSF membership list is the right choice here. It excludes a lot of people that actively participate in the development of Django. Many members within the DSF do not participate in the development of Django (the code base), as evident by many of the members comments.

If the goal is to minimise vote manipulation, then I think we should find another way to do so, or to insert protections where it's obvious that there is vote manipulation occurring.

Here are some ideas:

• Votes only count from members that have a minimum number of comments (participation threshold)
• Votes that have clearly been manipulated are pushed to the technical board for a decision
• Can we use the moderation tools of the list to block accounts that have been used for vote manipulation, and moderate away the vote?

[collinanderson]

Or maybe it could be limited to people who have commits in Django?

[claudep]

To be clear, I didn't propose to replace the Framework team by the DSF membership, but using the DSF membership as a requirement to be a Framework member. So if someone is actively participating in the development of Django, he shouldn't have any problem joining the DSF first.

[claudep]

So to be member of the Foundation, you have to be co-opted by other members, and to be member of this Framework team with voting right on the Django code, which is still the core of Django, you could freely join. I don't follow that logic. Aren't we mixing the ability to contribute to Django with the ability to take decisions for Django?

[LilyFoote]

I just noticed you mentioned vouching by a DSF member in your original post - I think that's a good compromise actually.

[aaugustin]

There was about 1000 subscribers to django-developers last I checked (years ago). Re-enrolling all these people through a validation process isn't realistic.

In a previous comment, it was suggested to use the Technical Board as a safeguard against vote manipulation. That could lead to an interesting situation if the current Technical Board said the election of a new Technical Board was manipulated. There's a word for that: "coup" ;-)

The current proposal suggests oversight by the DSF board, which seems like a better choice, as vote manipulation would be a community concern rather than a technical concern. It mentions explicitly that the DSF board could challenge bad faith registrations — typically a bunch of people with no history of contributions to Django signing up for the mailing list around election time would fit that criterion.

---

This is open-source. The regulation mechanism is forking. If two groups of people want to take Django in two irreconcilable directions, the result is a fork. There isn't much we can do to prevent that besides fostering a healthy community.

If thirty people we've never heard about (or bots) show up and try to take Django into a direction that we (the set of people reading and discussing this) aren't comfortable with, I think we'll manage, one way or another. We have all the keys to everything. We'll use common sense.

That's a nicety of keeping humans in charge rather than so-called "smart contracts" :-)

[ubernostrum]

We could use the DSF membership, but we'd have to completely rearchitect DSF membership to do that. Using the set of people who've expressed interest by joining the dev list, with the DSF Board as a check on people trying to sign up tons of troll accounts to vote, seemed the simpler option there.

[shaib]

In reply to: "There was about 1000 subscribers to django-developers last I checked (years ago). Re-enrolling all these people through a validation process isn't realistic."
We could automatically validate all the people who are on django-developers who are also DSF members. This would mean that anybody who wants to elect the technical board, and is a little less involved in the community, will need to ask to join (the electorate, not necessarily the DSF; and they'll need to be vouched for, of course).
Forking is painful. And I think we should consider a mechanism that doesn't invite attacks -- although the simplicity of complete openness does have its advantages.

[aaugustin]

@nemesisdesign https://groups.google.com/d/msg/dsf-members/yqnWGII63mI/LUqU-FJNAQAJ

I think dsf-members is a private group. You may not be able to view the discussion unless you're a member of the Django Software Foundation.

[timgraham]

In my opinion, "The Framework team shall keep in mind the need for sufficient time-zone coverage to ensure availability of at least one Merger during sprints at major events such as DjangoCons." isn't needed. I think that the goal of sprints shouldn't be to get a bunch of things merged, but to get tickets to "Ready for checkin" so that the patches can be more carefully reviewed after the sprint is over. Otherwise, it creates pressure on mergers which sacrifices quality.

[adamchainz]

Agree, there has never been a rush to merge stuff at any of the sprints I've been at and I don't think anything has suffered as a result

[jarshwah]

What has been useful at sprints are members that have a history of contributing to the code base that can help new devs setup environments and teach how to triage issues. I don't think this should fall to the merger role, but it's something we should consider in the future for sprints. Perhaps the DSF can put a call out for "helpers" for the major sprints that can take on this role?

[timgraham]

I'm not sure there's a need to limit the number of mergers. If we have a lot of good people willing to volunteer, then why not? The rationale for limiting the number of mergers should at least be explained.

Also, there's some value in having GitHub permissions to a repository besides pushing to edit. For example, closing spam submissions, editing the pull request description, or retitling pull requests to follow the commit message format.

[jarshwah]

Regarding github permissions, this is something Mariatta has considered for the PEP she is writing to move issues to github: https://mariatta.ca/core-sprint-2018-part-2.html

So what we can do is to invite the current bug triagers as collaborators for CPython repository, give them that write access, and add branch protection such that only Python core developers can push / commit into the active branches.

Protect the release branches to a Mergers group.

[aaugustin]

The reason is that, without this restriction, there's a good chance we'll simply recreate the core team by giving access to GitHub to whoever had it and still wants it.

Being much more liberal in giving access to GitHub and protecting the master + release branches looks like an interesting alternative.

[ubernostrum]

What @aaugustin said -- the worry is if there's a large Merger team, it's just "Django core" with a different name. The key thing here is turning merging code into a role rather than a glamorous privilege, and moving decision-making to a much more open process.

[MarkusH]

How is "being a merger" and thus one of a "small set" less of a "glamorous privilege" than being part of core, which is significantly bigger than the "small set"?

[timgraham]

I think it makes sense to keep releasers as a separate team. It's non-trivial to set up all the permissions for someone to release Django, permissions which they may never use.

[timgraham]

I think using the DSF membership makes sense. That would eliminate the preregistration step in each election.

[timgraham]

I would think there should be some activity guideline for mergers (i.e. merging an average of x pull requests per month or something). Not necessarily a requirement since life happens, but an expectation of stewarding the privilege, especially if the group is intentionally kept very small.

My alternate proposal would be to let the mergers team manage itself. Contributors to Django apply to become a merge if they think they'll use that privilege and self-renew yearly if they're still active. Existing mergers are in the best position to evaluate new members since they're reviewing pull requests from all contributors and will have a good sense if an applicant follows Django's contributing guidelines without much guidance.

[jarshwah]

and self-renew yearly if they're still active

This is the kind of thing I was wanting to see. It'll allow inactive members to cycle out.

[aaugustin]

I'm in favor of self-management of each team because we haven't been very good at centralized management.

[shaib]

I suggest that the rules for removing Mergers should be set together with the rules for appointing them.

[nemesifier]

@aaugustin, I see. As someone who likes to follow the development from outside, I wanted to let you know it's not really clear why this is needed in the first place. I've seen great improvements in django in the last years and I didn't think something like this was necessary. I think making the reason for this proposals clear would help the community understand how to help out.

[frankwiles]

@nemesisdesign This change isn't intended to fix a problem with what has been developed, but specifically to help encourage more people to become involved in Django without the need for a fairly closed "core" team.

We hope that by dissolving "core" and making the process more open in general we can attract more people to become major contributors vs the current process.

[aaugustin]

The reason is that, without this restriction, there's a good chance we'll simply recreate the core team by giving access to GitHub to whoever had it and still wants it.

Being much more liberal in giving access to GitHub and protecting the master + release branches looks like an interesting alternative.

[aaugustin]

Is it a good idea to allow people who haven't committed to Django in a very long time to be elected to the technical board? Did you consider "at least 18 months and no more than 5 years" ?

[ubernostrum]

I want people to have commits in Django at some point in the past. I don't care as much if they're still generating commits today, if they're also contributing to the progress of the framework in other ways. Someone with a contribution history who hops into technical discussions on the django-developers list, for example, should be eligible even if they haven't personally contributed any code in a while.

[aaugustin]

I think we should also recognize contributions to significant third-party packages as a qualifying criterion for the technical board. Indeed, decision made by the technical board should consider the whole technical ecosystem. The perspective of an "outsider" can be useful.

[aaugustin]

There was about 1000 subscribers to django-developers last I checked (years ago). Re-enrolling all these people through a validation process isn't realistic.

In a previous comment, it was suggested to use the Technical Board as a safeguard against vote manipulation. That could lead to an interesting situation if the current Technical Board said the election of a new Technical Board was manipulated. There's a word for that: "coup" ;-)

The current proposal suggests oversight by the DSF board, which seems like a better choice, as vote manipulation would be a community concern rather than a technical concern. It mentions explicitly that the DSF board could challenge bad faith registrations — typically a bunch of people with no history of contributions to Django signing up for the mailing list around election time would fit that criterion.

---

This is open-source. The regulation mechanism is forking. If two groups of people want to take Django in two irreconcilable directions, the result is a fork. There isn't much we can do to prevent that besides fostering a healthy community.

If thirty people we've never heard about (or bots) show up and try to take Django into a direction that we (the set of people reading and discussing this) aren't comfortable with, I think we'll manage, one way or another. We have all the keys to everything. We'll use common sense.

That's a nicety of keeping humans in charge rather than so-called "smart contracts" :-)

[aaugustin]

I'm still unsure about the reasons for prohibiting these well-known techniques for reaching consensus.

[ubernostrum]

Mostly: if we're going to vastly increase the number of people who have a vote, we need to figure out how to manage it. Setting up the process so people only vote when they actually have a strong opinion (as opposed to a bunch of +0/-0/explicit abstain votes from people who don't have strong opinions) seemed like a way to do that.

[aaugustin]

Thanks for clarifying.

[adamchainz]

The "+0 / -0" votes always confused me when I started reading django-developers. I think it's a good idea to get rid of them.

[shaib]

Eh, "+0" can be a concise way to express an opinion, and I will be surprised if people stop using them just because a DEP said so.
I think we should rephrase this as "In the interest of making votes with many participants manageable, votes other than '+1' and '-1' will be ignored."

[hvdklauw]

+0/-0 only makes sense if there are votes that can only pass with a certain percentage of eligible voters having voted. As the voting happens on the mailing list and if you signed up you will probably read it within a week and thus be able to vote on it.

If a Merger want certain people to have a look at a proposal and vote on it there is still the option to contact them directly to ask them for their input/vote.

[aaugustin]

I'm in favor of self-management of each team because we haven't been very good at centralized management.

[aaugustin]

It seems consistent with the historically low turnover of the technical board.

[adamchainz]

I've been silent on this discussion but that's because overall I think it's a good idea. I am one of the more recent additions to django-core back in Nov 2016 but I feel like it's mostly been a badge than a role as I've contributed less since then. Also the only time I used my commit access to was to put my name on the core team page, I always leave PR's for our current Mergers (the Fellows) as that process works really well.

Thanks for all your work on this @ubernostrum et al

[carltongibson]

Ref "shepherd":

I'm volunteering for this role.

• @aaugustin

Hi @ubernostrum, @aaugustin. I was asked when this DEP would go through. Bar bandwidth, are there any blockers or areas which could benefit from help?

Thanks for all the effort!

[timgraham]

Since releasing Django requires pushing version bumps, tags, etc. to Django. I think it would simplify things to say that releasers are a subset of mergers. Or if you really want releasers who aren't mergers, then say that releasers use their push access only to release Django. I think requiring coordination among a merger and non-merger releaser adds unnecessary complexity.

[ubernostrum]

I went back and forth on this a bit, and I don't think it should be required that a Releaser is a Merger. There might be times when it requires coordination between a non-Releaser Merger and a non-Merger Releaser, but I want the flexibility to decouple the roles from each other for future-proofing purposes -- the fewer people we have to set up with permissions to push packages (especially as Django Fellows can rotate out), the better.

[shaib]

This does not contradict what Tim said -- his argument is that Releasers, whether they are mergers or not, should have the commit bit in order to simplify the release process. I concur.

[timgraham]

I think it would be helpful to explain why technical board oversight is needed here and what "determination of release readiness" looks like. IMO, the mergers are the ones following day to day development and would be in the best position to determine whether there are any blockers to the release.

For each release since 1.8, I've created a page like https://code.djangoproject.com/wiki/Version2.2Roadmap#schedule and we've always hit those dates pretty close.

IMO, the "release readiness" status is fairly obvious -- just look at the release blockers. It doesn't take a week to figure that out and meanwhile, new blockers might appear.

[ubernostrum]

The main reason behind this is the trend of ensuring no one person gets to make large decisions (even if they seem fairly obvious) about Django. Having feature releases be approved by the technical board isn't a huge burden, I think, and accomplishes the goal.

[timgraham]

I don't see release candidate and final releases as large decisions. Perhaps if you list the steps for "determination of release readiness" you can convince me that this is something that the technical board adds value by doing.

In my mind, the release that involves the most discretion is the feature freeze (alpha release). Since 1.8, I've always been able to get a consensus about how to proceed on the "status of X.Y release blockers" threads. There might be some value in detailing the responsibilities of a release manager unless you wish to eliminate that role. I think that role has also worked well for Python.

[carltongibson]

As I read this the ...prerogative to ask... is key.

If no-one asks then Tim's story will be the one in play. 99 times in a 100 that'll happen. But maybe one-time there is a need to ask, then we have a documented policy.

Is that a fair reading?

[shaib]

This seems odd, and very different from existing processes AFAIK. The last time I followed the release process closely, the release was made on a pre-determined date, provided that no blocker bugs existed. This is an objective measure and a proven process. Why change it?
As a side, "lawyerly" argument, and as noted by @timgraham , it seems that this section aims to change the role of Release Manager -- which, by definitions at the preamble, would be out of scope.

[ubernostrum]

There seemed to be a desire (from the first round of feedback) to define a "Releaser" role. I'm not super committed to doing it in this document, which is why the Releaser bits are pretty rough.

The part about asking the technical board to approve a release is mostly there to assuage fears that the more open general decision-making process could push out a really bad release, by allowing the technical board to put the brakes on and prevent it going out the door. Open to better ideas on how to do this.

[timgraham]

I think you could remove these N/A sections.

[shaib]

As @timgraham points below, Alpha probably shouldn't be grouped with Beta -- it is the "anchor" of the release cycle, the other phases follow almost mechanically.

[shaib]

In reply to: "There was about 1000 subscribers to django-developers last I checked (years ago). Re-enrolling all these people through a validation process isn't realistic."
We could automatically validate all the people who are on django-developers who are also DSF members. This would mean that anybody who wants to elect the technical board, and is a little less involved in the community, will need to ask to join (the electorate, not necessarily the DSF; and they'll need to be vouched for, of course).
Forking is painful. And I think we should consider a mechanism that doesn't invite attacks -- although the simplicity of complete openness does have its advantages.

[shaib]

Eh, "+0" can be a concise way to express an opinion, and I will be surprised if people stop using them just because a DEP said so.
I think we should rephrase this as "In the interest of making votes with many participants manageable, votes other than '+1' and '-1' will be ignored."

[shaib]

redundant ")" in the end

[shaib]

But if I read things correctly, a Releaser may serve on the Technical Board.

[ubernostrum]

Yeah, I still don't know exactly what I want to do with the Releaser qualifications.

In my head, I saw it as not necessarily having full commit privileges, so less of a potential conflict in having a Releaser on the technical board, but as I said in another comment the whole Releaser role is still pretty rough right now.

[shaib]

Every voter gets to vote for five candidates (as we do today)?

[shaib]

"Automatically triggered" here means "it's the DSF Board's responsibility to actually set things in motion", right? Perhaps it should be phrased more explicitly.

[MarkusH]

I don't think the DSF should trigger the vote. Why can't the technical board trigger it themselves?

[shaib]

I think that if a member of the Technical Board has managed to convince all other TB members and DSF Board members that they (singular they, the first member) should be removed, we shouldn't limit the grounds for removal. I would require only unanimous agreement on the reason for removal.

[shaib]

I disagree. The Mergers are not beaurocrats -- they are asked to apply skill, taste and judgment.

I would rephrase this more along the lines of "remove the prestige & status which had, in the past, been given somewhat arbitrarily, and replace them with roles which are more in line with work actually being done".

[ubernostrum]

There's a difficult line to walk here in trying to outline the role of the Merger, because the goal is literally to remove as much individual "skill, taste and judgment" as possible from the process. Otherwise we might as well just say "we're keeping everything the same, but the core team is now called the Mergers team and there are fewer of them".

In my ideal world, the Merger role would literally be "push the button when consensus is reached on django-developers to do so". I can't make that happen since requiring a formal discussion + vote on every single PR is unlikely to scale, but that's where I'm coming from.