Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

2025 proposal selection process #657

Open
wants to merge 9 commits into
base: main
Choose a base branch
from
180 changes: 180 additions & 0 deletions 2025/selection-process.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,180 @@
# Interop 2025 proposal selection process

Interop 2025 is an effort to increase interoperability across browsers
in key technical areas that are of high priority to web developers and
end users. This effort is part of the
[web-platform-tests](https://github.com/web-platform-tests/wpt) (WPT)
project — an automated test suite for web standards — and run by a
team of representatives from companies that make substantial
contributions to browser rendering engines (including Apple, Bocoup,
Google, Igalia, Microsoft, and Mozilla).

Please see the [Interop 2024 Dashboard](https://wpt.fyi/interop-2024)
and [Interop 2024
README](https://github.com/web-platform-tests/interop/blob/main/2024/README.md)
for the current iteration of the project.

Each iteration of the project starts with an open call for proposals,
followed by a selection and prioritization process by each
participating organization. Final decision on the inclusion of a
proposal is made by
[consensus](https://github.com/web-platform-tests/interop/blob/main/charter.md#:~:text=The%20team%20makes%20decisions%20based%20on%20consensus.%20A%20decision%20has%20consensus%20if%20it%20has%20support%20from%20at%20least%20two%20participating%20organizations%20and%20no%20opposition). For
Interop 2025, the timeline and the selection process is as described
below.

## Timeline

This is the general timeline for the Interop 2025 proposal selection
process. The dates are centered on the Interop team meeting (Thursdays).
Please note that the timelines are subject to change.

* Proposal submission window (3 weeks): Sept 19th, 2024 to Oct 3rd, 2024
* Proposal selection: Before December 19th 2024
* Scope of the Interop 2025 project published: the first half of February, 2025.

## Proposal selection process

The aim of the proposal selection process is to generate a shared
understanding of the merits of each proposal, and converge on a
decision about the most impactful areas in which we can improve
interop on the web over 2025.

As in Interop 2023 and 2024, details of proposal selection and ranking
remain participant confidential throughout the process. Final decisions
will be published as the Interop team.

### Prior to Proposal Submission

**Timeline**: Before September 19th 2024

The interop team will develop a non-exhaustive set of signals/criteria
that are considered important when assessing proposals. These will be
made available to proposal authors, and used later in the process when
championing and assessing different proposals. However they are not
binding on any specific participant i.e. participants are free to
consider additional criteria (such as individual resource constraints)
and weigh the shared criteria as they choose.

### Proposal Submission

**Timeline**: Before September 19th to October 3rd

Members of the web community, including participants, will be invited
to submit proposals for Interop 2025. These will take the form of
issues on the Interop GitHub repository. Issue templates and
documentation will be provided to guide proposers towards proposals
that meet the formal criteria (e.g. around testability) and which can
be assessed according to the agreed criteria.

Items from Interop 2024 which have not yet reached full
interoperability will be automatically resubmitted, and will go
through the process as for any other proposal. In addition new
proposals can be submitted that extend the scope of existing focus
areas (whether or not they have reached full interoperability).

As in previous years proposals can either be for focus areas or
investigations. For simplicity the remainder of this document will
only refer to focus area proposals, but this also covers proposals for
investigation efforts.

### Champion Selection

**Timeline**: October 3rd, 10th, 17th

Interop participants propose focus areas, consisting of one or more
proposals, that they would like to champion. Proposals will initially
be shared in time for the meeting on October 10th, and champions
finalised on October 17th.

Where more than one participant wants to champion the same focus area
proposal a single champion must be nominated for the remainder of the
process.

Champions and seconds (i.e. organizations that would champion a focus
area but were not the final selected champion) are recorded for each
proposal.

There is no assigned limit to the number of proposals one participant
may champion.

In case different focus areas are put forward with overlapping sets of
proposals, the participants are encouraged to work out a way to divide
the proposals into focus areas in a way that satisfies everyone. If
this is impossible then carrying forward multiple competing focus area
proposals is permitted, but final selections will only allow each
proposal to appear in a single focus area.

Any proposal without a champion will not be carried forward.

### Focus Area Proposal Refinement

**Timeline**: October 24th and 31st

Champions gather evidence of the impact of their proposals, using the
rubric developed earlier. Specific participants may also be asked
to provide (publicly available) information for all proposals where
that participant has ownership of the information (e.g. standards
positions, use counter data).

### Focus Area Proposal Presentation

**Timeline**: November 7th (double-length meeting)

Each participant gets 15 minutes of meeting time to present the
proposals they are championing, and make the case that they should be
considered a priority for inclusion in Interop 2025.

### Focus Area Proposal Ranking

**Timeline**: Before December 5th

Participants rank each proposal as P1 (highest priority) to P3 (lowest
priority), or veto.

There is no preset limit on how many proposals each participant may
assign each rank.

Initial rankings are submitted in time for the December 5th meeting.

### Focus Area Ranking and Selection

**Timeline**: December 5th, 12th, 19th

Focus areas with any vetoes are eliminated.

Proposals that have strong positive consensus (e.g. majority P1
rankings, no P3 rankings) are adopted. Proposals with a strong
consensus of low priority (e.g. many P3 ranking and no P1
ranking, or many P3 and P1 only from the champion) are dropped.

Proposals with mixed rankings are ordered according to the number of
P1 rankings, followed by the number of P2 rankings, followed by the
number of P3 rankings. Starting from the top of the ordered list
participants who ranked the proposal lower are given an opportunity to
explain their concerns, and suggest any adjustments that would change
their perspective. Proposal champions get final say on whether to
adjust their proposals in response to this feedback.

Following each meeting participants may adjust their rankings for the
remaining proposals up or down in response to discussions, any agreed
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed in today's meeting, there's some ambiguity here about whether adjusting up and down is the only way proposals end up being accepted or rejected, in particular as we approach the final decision on Dec 19th.

I think allowing for adjusted rankings in response to argument is good, and that will hopefully move a few things into the accepted state.

I would suggest that all remaining limbo proposals are individually decided by consensus, meaning two supporting and none opposed. This ensures that we have a decision on each proposal that reached this stage, and don't implicitly leave out the bottom N if we run out of time, for example.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've clarified at the end that we have to actually make a decision about any remaining proposals, that is they can't just be ignored. But I'm really hoping that we don't have to use that. If it's clear that everyone is "meh" about a proposal to the extent that we don't even discuss it that seems like a pretty clear sign that it's not important enough to be part of Interop.

changes to the proposals, or the overall composition of the project,
and the process repeats in the following meeting.

Before the end of the December 19th meeting there's a final decision
on any proposals without a clear resolution, and formal call for
consensus on the set of selected focus areas.

### Communication and Publication

**Timeline**: Jan 9th, Jan 16th, Jan 23rd, Jan 30th

Launch date is finalized. Focus area tests are labeled. Feedback for
proposal authors written.

### Launch

**Timeline**: February 6th 2025 (provisional)

The Interop 2025 dashboard is published. Proposal authors are provided
with feedback and the original GitHub issues are closed. Participants
publish announcements of the launch.