Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Some sort of governance/process #11473

Closed
gsnedders opened this issue Jun 12, 2018 · 24 comments
Closed

Some sort of governance/process #11473

gsnedders opened this issue Jun 12, 2018 · 24 comments
Labels

Comments

@gsnedders
Copy link
Member

This is something that's mostly been discussed on IRC at various points.

There's a number of issues here:

  • Where do policy decisions get made? (The status quo appears to be all over the place.)
  • How do we ensure interested parties are aware of where the decisions are being made? (The status quo appears to be random subsets of people.)
  • Is there anyone to whom you can appeal the decision?

cc/ @web-platform-tests/admins @annevk @domenic (who I think are the two who have brought this up most often?)

@foolip
Copy link
Member

foolip commented Jun 17, 2018

Where do policy decisions get made?

How do we ensure interested parties are aware of where the decisions are being made?

For both of these I would suggest a new repo, called "policy", "governance" or "meta", which is low-volume enough that it's possible to subscribe to it.

Is there anyone to whom you can appeal the decision?

@zcorpan and I discussed this a bit today. A steering group in the style of https://github.com/whatwg/sg might work, with the same expectation that the mechanism will always never be used, because people generally prefer to resolve their issues without escalation, where they have more control over the outcome.

The tricky part is how to create the initial group. Because it should be "boring" and effectively never used, having a big election or similar would make it seem more important than it is. My best suggestion might be to use the same procedure as when picking the logo, namely asking the top 10 wpt contributors vote if there are too many candidates.

@foolip
Copy link
Member

foolip commented Aug 16, 2018

@jgraham recommended that I watch How Rust Does Open Development. It's a long talk, but 2m12s is a brief history, 6m8s has the talk outline, and 46m1s has the main lessons.

The answer to WWIC? was an RFC process for "substantial" changes. Documented at https://github.com/rust-lang/rfcs. Not clear if there's a fast path when everybody agrees, I expect that'd be the case in WPT 90% of the time. I think a process like this would be good, a single place to monitor for "substantial" changes rather than relying on others poking you on big issues/PRs.

For a process like that to work we would still need a way to resolve disputes, i.e. a group of people who are trusted to make reasonable decisions. @web-platform-tests/admins could be this group, but it was created with no transparency, and I agree with @annevk (on IRC) that it's probably best to not conflate the two roles. Instead, we could create many teams or working groups as in Rust and have them decide in their areas, or perhaps have a single steering group intended as an infrequent escalation path.

There's also a lot of discussion on how to attract new contributors and help them advance to regular contributor and roles of leadership. This is an area we're frankly not doing very well in. This Week in Rust looks nice, I'd love a "This Week in WPT".

On planning larger arcs of work, I like the "slogans" idea, where the goals for a whole year are condensed into single short slogans. Currently we have no shared roadmap at all for WPT, only things like Ecosystem Infra 2018 OKRs which represent just what one company is up to.

@jgraham
Copy link
Contributor

jgraham commented Aug 16, 2018

After watching that talk and thinking abou the different requirements of wpt compared to Rust, I formulated a proposal. I'm not sure it's the right end point, but it might be a useful starting point for discussion. It notably doesn't cover people management issues like who enforces the CoC and how.

Goals

Have a system of governance for the project that

  • Offers cross-organisation leadership over project direction and priorities

  • Allows the community to follow the wpt work, and have input into the
    overall direction and into specific technical questions.

  • Provides a clear escalation path and decision making process both
    for technical and project management issues.

Influences

  • Heavily influenced by the Rust
    project

    which in turn takes inspiration from TC39 amongst others.

Proposal

  • Nominate a "core team" who are responsible for final decsion making
    and organisation of the project. The team should consist of ~5
    people(?) representing at least 3 different browser engines. The
    process for selecting/replacing core team members is by agreement;
    not by voting.

  • The core team may appoint further subteams as requied (for example
    one could imagine a team responsible for wpt.fyi or infrastructure
    or whatever). Each subteam should contain at least one member of the
    core team.

  • The core team is responsible for

    • Setting and communicating the overall priorities and direction
      of the project

    • Taking overall responsibility for shared infrastructure (GitHub,
      any shared password, etc.)

    • Being final arbiter on technical decisions

    • Making changes to the project goverance.

  • The core team may delegate day-to-day running of any part of the
    project to subteams.

  • In order to set and communicate priorities the core team commits to
    publishing a blog post periodically, setting out the priorities and
    direction of the project for the following period.

Making Changes

User-facing changes to the project require an issue in a special
repository that will form a "RFC". Examples of changes that might
require such an issue include:

  • Adding a new test-writign API to wptserve

  • Adding new features to testharness.js

  • Adding new test types or adjusting how tests are evaluated
    (e.g. adding fuzzy reftest matching).

There are several reasons to require an RFC-type issue in these cases:

  • Allows people to follow substantive changes to the project, in a
    setting that is much lower volume than the main wpt issue tracker.

  • Ensures that changes can meet the needs of all participants in the
    project before they are rolled out.

It is highly desirable that the process remain lightweight; compared
to a programming language decisions made for wpt are generally much
easier to reverse. Therefore in the case of no substantive
disagreement changes may be assumed to go ahead after a reasonable
period (e.g. 1 week).

In the case that an issue generates substantive discussion it is
assumed that the proposal will change in accordance with that
discussion. If participants are unable to agree on the correct
approach for a change — or whether the change may be made at all —
then the core team (or relevant subteam) must make a decison. The
exact mechanics for this are unspecified, but they must only consider
arguments that have already been made on the RFC issue.

@plehegar
Copy link
Member

I like the direction. Some comments below:

I would to make the "core team" to be at least 6 if we're keeping a requirement to have at least 3 browser engines. With only 5 and the requirement to have 3 browser engines, you're preventing the project to be independent from browser engines, which was always important in my mind to have on the paper, even if the reality is different (due to current major contributors). Having the potential to have some independence is important imho.

The scope of WPT, as a project, shouldn't be restricted to "unit tests for browser engines" imho. The overall project is about moving the Web forward using tests. I grant you that the current WPT repo achieves a lot for that. But I could imagine for example a separate repo called web-platform-tests/wpt-app-tests, which would focus on Web applications testing (like specific UI scrolling pattern/practices). So, those wouldn't be necessarily integrated into the browser engine implementer workflow. The core team overall goal should be bigger than just webplatform-tests/wpt. As such, in your list of potential changes requiring a RFC, I would add an another example something like "Adding a new repository for application-level or performance tests"

@plehegar
Copy link
Member

btw, for the core team, what type of agreement did you have in mind?

@foolip
Copy link
Member

foolip commented Sep 28, 2018

I've put together https://github.com/foolip/rfcs as a strawman for how an RFC process might look.

btw, for the core team, what type of agreement did you have in mind?

That question was for @jgraham, but this is still TBD in my RFC repo. I think the core team would also be the group that somehow resolves RFC that do not reach a consensus.

@foolip
Copy link
Member

foolip commented Sep 30, 2018

@jgraham, I've incorporated #11473 (comment) into https://github.com/foolip/rfcs by referencing a core team which does not exist yet, and using some of your phrasing.

@foolip
Copy link
Member

foolip commented Sep 30, 2018

As for the core team, I agree with @plehegar in #11473 (comment) that the core team would manage web-platform-tests/*, not just web-platform-tests/wpt. I don't think it's important that we have rules about what number of members have which affiliation, and I think 5 is a reasonable group size.

The main challenge I see is how to populate the initial group. One idea is a vote among the top 100 wpt contributors, but I think that would suggest more prestige than we intend. But just coming to a silent agreement would also not be great.

So, the procedure I'd suggest is that we find a trusted volunteer who does not themself intend to be a member to do the following:

  • Announce on public-test-infra that we're looking for up to 5 core team members, asking people to self-nominate or nominate others.
  • If there are more than 5 candidates, the trusted volunteer and the candidates discuss to see if there is a natural way to trim the list. (For example, more than one candidate from the same company might not be necessary.) As a last resort, the trusted volunteer gets to make a suggestion based on any criteria or method (possibly voting) they like.
  • The group of 5 is suggested on public-test-infra, with a 1 week period to object.
  • If there are no objections, https://github.com/orgs/web-platform-tests/teams/core is created.

@web-platform-tests/admins and others, would you find that procedure acceptable?

@sideshowbarker
Copy link
Contributor

@web-platform-tests/admins and others, would you find that procedure acceptable?

Sounds good to me

@marcoscaceres
Copy link
Contributor

top 100 wpt contributors

You might want the top 100 in last 12 months maybe? From the full list, lots of people are inactive.

@foolip
Copy link
Member

foolip commented Oct 1, 2018

That might be better, yeah, although I hope no voting will be required. I'd leave that decision to the trusted volunteer.

@Ms2ger
Copy link
Contributor

Ms2ger commented Oct 1, 2018

@web-platform-tests/admins and others, would you find that procedure acceptable?

SGTM, but maybe we can allow more than a week to object. We've gone without a core team for the best part of a decade, so I don't see a reason to rush it now.

@foolip
Copy link
Member

foolip commented Oct 1, 2018

Sure, we can give it a month, that would be fine.

@jgraham
Copy link
Contributor

jgraham commented Oct 1, 2018

I think that procedure is acceptable, although I was imagining something less complex.

Appointing a trusted volunteer is challenging for theoretical and practical reasons. The theoretical reasons are just that the appointment of such a person is itself a fiat decision that needs to be made by someone or some group of people. Therefore it doesn't really reduce the number of meta decisions that need to be made, just causes a small amount of indirection. Practically finding a person who both has the relevant context and is neutral is hard. However if people feel the complexity is adding transparency then I'm happy to go ahead with that part of the plan.

@foolip
Copy link
Member

foolip commented Oct 1, 2018

Alright, I'll first try to find a volunteer, if I can't then obviously we're stuck at step 1. I agree that picking a volunteer is also a fiat decision, although it is one that's open to objection by anyone.

@gsnedders
Copy link
Member Author

Doing it by voting alone seems likely to end up with @foolip, myself, and @jgraham among others on the core team; in a sense that's less than ideal because it means the majority of users of the project are essentially underrepresented (how many tests for the three of us write?). There's also the question of whether we want to encourage those currently less involved to become moreso.

@marcoscaceres
Copy link
Contributor

marcoscaceres commented Oct 1, 2018

Reading the responsibilities that @jgraham listed above, it makes sense that you, @foolip, and @jgraham would be on the core team. If you need a voice that represents test writers, there seem to be very good people in the top contributor list you could ping (e.g., Luna, Anne, Domenic, me, etc.)

@foolip
Copy link
Member

foolip commented Oct 16, 2018

@jgraham has sent out web-platform-tests Governance to public-test-infra, drafted by him, @marcoscaceres and me. The process is a little bit different (simpler) than what I described in #11473 (comment), so please read there and if you are interested fill out the self-nomination form.

@tobie
Copy link
Contributor

tobie commented Oct 16, 2018

Don't you want to extend this to the end of TPAC to make sure everyone sees it? Just a thought.

@foolip
Copy link
Member

foolip commented Oct 16, 2018

If someone shows up at TPAC that missed this discussion/announcement and would like to self-nominate, then sure, we'll include them in our effort to form a core team. But we'll want to do the "If possible a mutually agreeable decision is made from the pool of nominees" step during TPAC, so I don't think we should wait until the end to close the self-nomination form.

@annevk
Copy link
Member

annevk commented Oct 16, 2018

I got this clarified: there will be a 1 month objection period following any decision about who the core team should be with objections handled by @marcoscaceres: https://lists.w3.org/Archives/Public/public-test-infra/2018OctDec/0009.html. So whatever happens at TPAC is subject to scrutiny. This seems fairly reasonable to me.

@foolip
Copy link
Member

foolip commented Dec 6, 2018

@jgraham has announced the provisional membership of the initial core team and there is now a 1 month period in which objections to these members may be raised:
https://lists.w3.org/Archives/Public/public-test-infra/2018OctDec/0017.html

@tobie
Copy link
Contributor

tobie commented Dec 6, 2018

This team looks amazing. Congrats to all involved!

@gsnedders
Copy link
Member Author

We now have a process and an RFC repo, so we can close this!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

9 participants