add new feature proposal process

[samuel-oci]

Streamlining the feature proposal process for OpenSearch

Introduction:

OpenSearch is a fast-growing community-contributed project and currently has around 600+ contributors contributing to 110+ public GitHub repositories on a day to day basis. Open source contributions can take various forms, such as: Code contributions: writing, modifying, or optimizing source code to add new features, fix bugs, or improve performance.

This document will focus on streamlining the feature proposal process for OpenSearch. Currently the contributors who are interested in developing a feature create a GitHub proposal on the respective repository and submit it for community review. The maintainers of the repo reviews the proposal along with the community in a public forum and provide their inputs as needed.

Even though OpenSearch has a standardized GitHub proposal template, it doesn’t always suit the needs of the requester hence the feature requester tends to create a proposal that adheres to their needs. The proposal is also used as a placeholder for capturing discussions, reviewing design, tracking the tasks spanning across multiple versions, discussing open questions thereby creating ambiguity on the finalized scope of the tasks associated with a feature pertaining to specific release.

Goal:

The goal of this document is to establish a seamless process for submitting a feature proposal that underlies the OpenSearch project’s commitment to the following principles of development:

• Open source like we mean it
• A level playing field
• Made with your input
• Open to contribution

When successful, this new process will provide:

• Simplicity in suggesting and adding features
• Assurances to contributors that their contributions will be evaluated
• A trackable format that can be centralized for at-a-glance, project-wide proposals
• Clear ability to track a feature at release version granularity
• Open communication and discussion on upcoming features
• Clear workflows for maintainers

Issues Resolved

LC meeting agenda notes

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[dblock]

I like the spirit of this proposal.

The .github repo generally has documentation for what has agreed upon. How do you feel about turning this into a new FEATURES.md that has a section called Proposals (and other sections about when a feature is built, etc.)

Think about someone reading FEATURES.md to understand the process to propose a feature.

[jkowall]

This looks good to me; however, there are many details towards the end around the CI system which seem out of place. While we don't specify the testing and other criteria in the suggested improvements. I also noticed these details are missing in https://github.com/opensearch-project/.github/blob/332448b66491e50762ba1793db3b04176a7ac6fa/CONTRIBUTING.md which they should likely be in.

[samuel-oci]

This looks good to me; however, there are many details towards the end around the CI system which seem out of place. While we don't specify the testing and other criteria in the suggested improvements. I also noticed these details are missing in https://github.com/opensearch-project/.github/blob/332448b66491e50762ba1793db3b04176a7ac6fa/CONTRIBUTING.md which they should likely be in.

@jkowall are you referring to the flow at the very bottom (High level flow)? This one is mostly there to give an idea of the end-to-end process that needs to take place for the feature to go live. Is there a particular section/paragraph that you would like to change?

[samuel-oci]

I like the spirit of this proposal.

The .github repo generally has documentation for what has agreed upon. How do you feel about turning this into a new FEATURES.md that has a section called Proposals (and other sections about when a feature is built, etc.)

Think about someone reading FEATURES.md to understand the process to propose a feature.

@dblock sounds good to me, I moved it to FEATURES.md

[dblock]

The .github repo is to document the current process and piolicy, so FEATURES.md should document the proposed process succinctly that one can follow, not discuss the goals or the pros/cons of the process. Could you please remove those parts (move them to the issue) and boil this down to the absolute minimum as if this was the current process?

[dblock]

@samuel-oci for DCO make sure to amend commits with -s.

[anastead]

This looks good to me. +1

[samuel-oci]

The .github repo is to document the current process and piolicy, so FEATURES.md should document the proposed process succinctly that one can follow, not discuss the goals or the pros/cons of the process. Could you please remove those parts (move them to the issue) and boil this down to the absolute minimum as if this was the current process?

done, all commits are amended for DCO

[dblock]

I have some cosmetic/typos requests below.

Beyond that, I am personally not comfortable with the very directed, heavy handed, and detailed design document proposal. I would like to get more opinions.

That said, I do think we should have a feature proposal document. Is there a trimmed down version of the text here that matches what we do today (opening issues, having some structure and format to them, meta issues), and we can add more/new steps in subsequent additions?

[samuel-oci]

I have some cosmetic/typos requests below.

Beyond that, I am personally not comfortable with the very directed, heavy handed, and detailed design document proposal. I would like to get more opinions.

That said, I do think we should have a feature proposal document. Is there a trimmed down version of the text here that matches what we do today (opening issues, having some structure and format to them, meta issues), and we can add more/new steps in subsequent additions?

@dblock what's the concern regarding the guideline for design proposal?

[dblock]

@dblock what's the concern regarding the guideline for design proposal?

It's very prescriptive and a big jump from the process hundreds of people follow today. Can we reduce this PR to the non-controversial parts and then discuss the controversial ones one-by-one?

[samuel-oci]

@dblock what's the concern regarding the guideline for design proposal?

It's very prescriptive and a big jump from the process hundreds of people follow today. Can we reduce this PR to the non-controversial parts and then discuss the controversial ones one-by-one?

@dblock Are you suggesting to just say we need a design doc without specifying any particular way for the design doc to look like? And leave any format for the design doc to be discussed later?

[dblock]

@dblock Are you suggesting to just say we need a design doc without specifying any particular way for the design doc to look like? And leave any format for the design doc to be discussed later?

I suggest removing the design document completely from this PR. I would cut everything after, and including "The criteria to continue with a feature proposal are as follows:", and edit the text to be a lot shorter as a first cut at FEATURES.md. We can then add to it parts of the rest of this proposal, and debate whether a design document is the right mechanism in a separate PR.

[elfisher]

Latest updates look good to me.

[dblock]

Lots of cosmetic changes and a bigger one to extract the template into an actual template.

[dblock]

Mostly cosmetic stuff:

• Can you please Capitalize Sections Like This (that's the format with all the others)?
• In README it should match the doc title, "Feature Proposals"
• A section called "Appendix" is odd looking, and it says "Meta-issue format", but this is not a meta-issue format, it's a feature request template. Maybe just link it in the text where you mention it and remove the Appendix section?
• You suggest a label, feature-proposal, but we don't have that label anywhere. We have some feature. Most repos use the GitHub default Enhancement everywhere, is that acceptable or should we go and create a label called feature-proposal in all repos? How is it going to interact with Enhacement?
• You left some capitalized META.
• The table doesn't match the text, there's still Approval of feature proposal. Feature proposal owner is capitalized differently than the rest.

Sorry to be a pest. This stuff is read by hundreds of people, so I think it's important to sweat the small stuff.

[samuel-oci]

• The table doesn't match the text, there's still Approval of feature proposal.

@dblock can you elaborate on where is the gap here? we need maintainers/contributors/community to make sure the proposal meets guidelines and standards. Is it the process or the wording of this you would like to change?

[dblock]

• The table doesn't match the text, there's still Approval of feature proposal.

@dblock can you elaborate on where is the gap here? we need maintainers/contributors/community to make sure the proposal meets guidelines and standards. Is it the process or the wording of this you would like to change?

We say The maintainers of the repo decide when and if the proposal meets the criteria., but we don't say "approval" anywhere, which seems like you need to wait for maintainers to get approval to move forward. There's maintainer engagement, and maintainers make some decisions.

[dblock]

• For DCO, the Author: Samuel ... part is missing a .com, which should be fixed in the local git config.
• The section "Writing the code" should be "Writing the Code", like e.g. https://github.com/opensearch-project/.github/blob/main/COMMUNICATIONS.md#workspace-channels, which is "Workspace Channels", same for other sections. Don't forget to update the TOC.

[dblock]

I extracted parts of this into #210, please review.