Request for Comment (RFC) Process #222
Conversation
Automated Review URLs |
|
This pull request has been mentioned on Image.sc Forum. There might be relevant details there: https://forum.image.sc/t/ngff-rfc-process-proposal-draft-now-available/90181/1 |
|
Happy new year, everyone! Fantastic write up @joshmoore! I think this RFC process will help to drive innovation in the NGFF space. While reading through the proposal, 2 things came to my mind:
|
There was a problem hiding this comment.
Wow, spectacular @joshmoore !
I love 😍 :
- The well-defined process!
- Building on the process of arguably the most successful and impactful efforts for interoperability, the internet.
- Multiple stages for incremental evolution and improvements.
- A process that promotes a "defaults to yes" as opposed to "defaults to no" so we can progress and effort is not wasted.
- Artifacts that result in a clear record of what has been proposed and discussed.
- The diagram that provides a succinct high-level overview.
- The time limits at stages.
- A recognition in the process that full consensus is not what will allow the community to move forward, which is what we all really want.
- The process essentially utilizes the same GitHub pull request-based system, but formalizes the phases and content, and ensures drafts and reviews and responses are more easily navigated.
Regarding possible areas of improvement, there are a few tweaks we could make to have the intended effect. In particular, I am thinking of dynamics common in specification development where:
- Changes are prematurely or unjustifiably suppressed because reviewers do not personally have interest in or and understanding of the change.
- Strong-willed, arm-chair speculation on how a change should work without experience in implementation or usage.
- Overly complex proposals that are difficult to implement.
- Changes are rejected too early in an incubation stage.
Another successful community process to cite that I think has lessons to learn from, also a W3C project, is the WebAssembly process.
In particular, I think it is helpful to specify expectations on different degrees of implementations throughout the process. As we know, speculation on what should work well or how it should work is often clarified or modified during implementation and usage. There is also an iterative, evolutionary process to a final specification and implementation for good specifications. It is also helpful to clarify the specifics of a proposal to reviewers with a concrete example. And concerns about the impact of the complexity of a change on practical deployment are often resolved with reference implementation(s). By evolving an implementation with the spec, even if the implementation lags a bit, this helps avoid inconsistencies, which can be problematic even if it is just schema / documentation inconsistencies.
The following changes could help address these issues with this in mind:
- Reserve input from Reviewers and Commenters until a spec has gone into the RFC stage (I am afraid we may not see a practical difference with the status quo otherwise).
- Before exiting the DRAFT phase, request some partial implementation reference to link to.
- Before exiting the RFC phase, a full implementation with a test suite is required.
- Before entering the SPEC adopted status, there must be at least two implementations for the spec in the community.
rfc/x/index.md
Outdated
| The DRAFT phase begins when **Authors** propose (D1) a new idea and | ||
| subsequently gather support (or "socialize") the idea (D2) before opening a PR | ||
| (D3). Open PRs can be discussed on GitHub. Any clarifications that are needed | ||
| can be requested by **Editors**, **Reviewers** or **Commenters** (D4) which may |
There was a problem hiding this comment.
In keeping with the last paragraph, i.e. the DRAFT stage promotes ideation without premature criticism and procedural indecision, it would be helpful to remove **Reviewers** and **Commenters** here and invite **Commenters** participation in the RFC stage.
rfc/x/index.md
Outdated
|
|
||
| This brings a critical, and iterative, decision point (R6). If a "Reject" | ||
| recommendation remains, then the RFC is closed. The text remains on the | ||
| specification pages for posterity. If sufficient (TODO: define minimal, etc.) |
There was a problem hiding this comment.
Sufficient: two released implementations?
I definitely agree that something like this is needed, but I also know that Thar Be Dragons. I'm inclined to say that should follow as a fully fledged RFC itself, but it's worth considering if we can make an initial stab at it.
My feeling is that it shouldn't be part of the initial RFC PR because that makes merging that PR much more difficult. Currently the change happens at
Thanks for the link. I'll read and respond.
It's a good point and I think reflects how GitHub has not been ideal for what we need. Are you proposing explicitly removing D4? Two options I considered to that end were: using GH locking mechanism to prevent
Interesting. That's quite the bar but I can definitely see the benefit.
Evaluating the "full"ness and the "suiteness" of the tests may be an issue, but I concur. I could also see adding "...a released implementation..." to the requirement.
I've left your comment on line 215 open for further discussion. Thanks, both! |
@joshmoore yes, I agree with this approach. That is, keep D4 and limit the GitHub pull request review discussion to minimal clarification on what is being proposed. This could come from the Editor. It would be helpful to also set the expectation that PR comments on how a specification is implemented or what could be proposed instead from Reviewers or Commenters should be come in the RFC stage, and the Editor will direct these comments to the RFC stage when appropriate. I am not sure if we want to invite limited clarification questions on what is being proposed from Reviewers and Commenters because it will be tempting to derail into more comprehensive discussions. These suggestions and discussion on the content of RFC 0 should probably be made once RFC 0 reaches the RFC Stage, though! 🤷 |
😄 Exactly. So we clarify the few remaining TODOs, then move this forward for fuller reviews ❤️ |
|
|
||
| At the **Editors** discretion (D5), the PR can be merged at which point it | ||
| becomes an RFC or closed if there is no interest in progressing with the | ||
| proposal. In the latter case, **Authors** can use any feedback to open a new PR |
There was a problem hiding this comment.
The diagram.png doesn't mention that the DRAFT PR gets merged or closed at D5. Only approve Yes/No. The PR persists state doesn't indicate that the PR is closed.
Is there a point at which the DRAFT PR loses the "DRAFT" status and becomes a "ready to review" PR, or maybe DRAFT PR is a PR to DRAFT and RFC, rather than a PR with "DRAFT" status on github?
It wasn't clear to me from the diagram whether the RFC section was reviewing an open PR, but from reading the text here is appears not.
Who is responsible for D4? It's the only block that isn't assigned to a stakeholder role. I assume it is REVIEWERS but are they invited or notified about the PR by EDITORS?
There was a problem hiding this comment.
The PR persists state doesn't indicate that the PR is closed.
👍 I'll change "PR persists" to "PR closed"
Is there a point at which the DRAFT PR loses the "DRAFT" status and becomes a "ready to review" PR
The GitHub "draft" state doesn't currently play a role in the RFC, but I can see that that could cause confusion.
It wasn't clear to me from the diagram whether the RFC section was reviewing an open PR, but from reading the text here is appears not.
Correct. I'll include "EDITORS ... merge PR" in R1.
Who is responsible for D4?
Anyone in the community, but it won't be requested of anyone.
Thanks for the read, @thewtex! I could definitely see lifting some of the formatting (e.g., phases) or phrasing (e.g., champion) for this proposal. The barrier I see to making more use of the process in general is the amount of dedicated capacity that I imagine it takes. We're not yet in a position to get that number of decision makers to synchronously come to consensus, not to mention to get them colocated physically in the Bay Area! 😄 It's a given, though, that this RFC process will need to evolve as the NGFF community changes. Assuming the number of individuals dedicated to driving the specification forward starts to increase, let's come back to this model and review. I've now taken this PR out of GitHub draft mode and expect to merge it soon and solicit reviews. I'm currently trying to finalize RFC-0 with a summary of the original consensus model, but don't want to block too long on that more aesthetic addition. |
joshmoore
changed the title
No offense to the Bay Area, but I would prefer Napari. Or Kaibu 😄 #mergeit |
|
Hey @joshmoore . I think this is great. I agree with @thewtex 's list of strengths of this RFC above. I think there are some details to be worked out, but I think this is clear enough to move to the next stage. The fact that there is already an RFC drafted (#227) is a fairly strong endorsement in my eyes. I agree with your plan to merge and solicit reviews! 🚀 |
😄
Thanks all. Merging for the rebasing of #227, and I'll push RFC-0 separately. |
Request for Comment (RFC) Process SHA: 24499ae Reason: push, by @joshmoore Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
|
Nice! Thanks @joshmoore ! |
Fix #132 (image.sc announcement)
This is a proposal for the use of the RFC model,
established by the Internet Engineering Task Force (IETF), for decision making
within the NGFF community. As a proof-of-concept, the PR uses the RFC process
that it itself is proposing in order to demonstrate that process. The description
here on GitHub will be kept to a minimum in favor of reading the RFC text.
One possible order of review would include:
template.md: Read the template document which describes each of the sections
in an RFC in approachable language (largely based on the Hashicorp model from
[https://works.hashicorp.com/articles/rfc-template]). A review of the
sections from templates in several RFC-using communities can be found in
a google spreadsheet.
diagram.png: Take a quick look at the process diagram linked below to get an
overview of the number of steps and phases, but don’t be too concerned about
the details since they are described in the RFC.
RFC-x.md: Read the RFC itself which has all of the sections described in
the template.md and walks through each of the phases and states, describing
all terms.
Read some of the other proposed RFCs (once they are ready)
Expectations
As mentioned in the RFC text, not all PRs are expected to go through the RFC
process. In fact, the number that are in the process at any given time is
likely limited to a handful at most. (These numbers have been omitted from the
RFC text since they are estimates and likely to go stale.)
Open questions
As a WIP, the RFC text currently has a number of TODOs listed throughout that
need resolution before this PR will be ready for review. Additional open questions
that could use consideration before a closer review is possible include:
The times suggested in the drawing are initial suggestions and up for
discussion (as is the general flow).
It is not yet defined which sign-offs are required for this RFC! Work on
that will begin in the new year. There will likely also be a separation
between the reviewers of technical PRs and of process PRs.
In IETF RFC, texts are not edited once they are adopted (see rfc2026).
If that model is followed, RFCs opened to modify existing processes will need
to copy the entire text of the original RFC. This is likely more complicated
than we would like. What modifications, however, are permissible on published
RFCs? Do they need versioning?
How do we gauge the impact on implementors? This could likely use
clarification in the SPEC phase.
Do versions of the full spec get their own RFCs? How do alpha/beta releases
work?
In some cases, these questions may be moved to the "Future possibilities"
section of the RFC to be answered by future RFCs.
Open actions
reference to this page. Then when other RFCs are added, that central location
can be updated (and we can decide whether or not we update the RFCs
retroactively, or perhaps only in the metadata)
move it to "Future possibilities"