Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add PRR questionnaire to KEP template, require for targeting #1620

Merged
merged 1 commit into from
May 7, 2020

Conversation

johnbelamaric
Copy link
Member

This PR adds the PRR questionnaire as a formal part of the KEP, and makes it required when targeting the enhancement at a release.

A follow up PR in k/community will remove the questionnaire from there and point folks here.

/sig architecture
/sig release
/assign @wojtek-t @deads2k @justaugustus

@k8s-ci-robot k8s-ci-robot added sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. sig/release Categorizes an issue or PR as relevant to SIG Release. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. kind/kep Categorizes KEP tracking issues and PRs modifying the KEP directory labels Mar 18, 2020
@johnbelamaric
Copy link
Member Author

/assign @jeremyrickard @mrbobbytables

- [ ] (R) Design details are appropriately documented
- [ ] (R) Test plan is in place, giving consideration to SIG Architecture and SIG Testing input
- [ ] (R) Graduation criteria is in place
- [ ] (R) Production readiness review [questionnaire](production-readiness.md) completed and approved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing that is not clear for me is who is allowed to approve it. Do we fully delegate to KEP approvers?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

good point. I don't think so. we can use an OWNERs file with a regexp.

Process-wise, sig-scalability probably needs to approve the scalability section. In the PRR KEP we discuss a team similar to API review team. Perhaps we need to implement that before merging this.

@@ -0,0 +1,139 @@
# Production Readiness Review Questionnaire
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given you move it here, we probably should remove it from the community repo, to avoid maintaining the same thing in two different places.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, that's the plan. Different repo though so it will be a separate PR.

@justaugustus justaugustus added this to In progress in Enhancements Subproject via automation Mar 18, 2020
@justaugustus justaugustus added this to the v1.19 milestone Mar 18, 2020
@justaugustus
Copy link
Member

/hold for my review

@johnbelamaric -- On an initial glance, I'm wondering if there's a good reason for the PRR to not just be a section in the README.

Apologies if I'm lacking context from another conversation.

@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Mar 18, 2020
@johnbelamaric
Copy link
Member Author

@justaugustus that was the initial thought, but I split it out for two reasons: 1) it is long; 2) it should have a separate set of approvers. But that latter process is ill defined at the moment.

@justaugustus
Copy link
Member

@johnbelamaric --

  1. it is long

I think this is fine, especially if it means that there will be more eyes on the PRR section.

  1. it should have a separate set of approvers. But that latter process is ill defined at the moment.

Having a separate approver set is impossible today with the way OWNERS work.
We could move PRRs to their own directory, but I don't necessarily like the idea of having them too far from the KEP.

We could have the PRR approver added to approvers set instead and have that be a requirement for status change?

approvers:
  - "@johnbelamaric" # PRR Approver

@johnbelamaric
Copy link
Member Author

Ok, for some reason I was thinking we could do regexp for file names in OWNERS files. Not sure where I got that idea...

I don't want to move it to its own directory either. I think your suggestion is good, I'll update the PR.

@johnbelamaric
Copy link
Member Author

Ok, I added it directly, and also mentioned that it needs a PRR approver. But we should hold this until all that is set up.

cause increased failures in production. See more in the PRR KEP at
https://git.k8s.io/enhancements/keps/sig-architecture/20190731-production-readiness-review-process.md

The KEP must have a approver from the PRR review team.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Regarding the tooling, I have no idea how the existing tooling works, so I may be completely wrong, but I think it hopefully should be a small thing to do. How I envision this to work is:

  1. add another PRR_OWNERS file (or extend the existing one - thanks @johnbelamaric for this thought) [we can probably start with just top-level one
  2. introduce prr-approved label (naming to be done) based on /prr-approve comment from PRR_OWNERS (hopefully that's only a matter of configuring existing tooling?)
  3. change tide for this repo to require both approve and prr-approve labels?

As @johnbelamaric pointed out we will probably need to also auto-assign prr-approvers. Though I think that as a first step we can probably ask sig-leads (who are the only approvers for now) to add someone manually.
The remaining thing is that not every PR should require PRR-review. But OTOH, those trivial ones will probably be easy to review and will not add significant overhead.

@spiffxp @cblecker - does that above makes any sense from the tooling perspective? How much work would it be to make that happen? [If you're not the right people to answer, can you route it to the right ones?]

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I'm a fan of a coding something specifically into prow for this purpose. I agree we're seeing cases for more flexible review/approval workflows than what we support right now, but it's unclear what the right path forward is, and nobody has improving this as a priority. Need some time to think about this. @cjwagner @fejta do you have any ideas?

A mechanism you could try to use today is to force KEPs going to release to use a directory based approach (I'm not clear if that's the standard now or if we're still putting everything in one file), put the PRR in subdir of the KEP, and use a no_parent_owners directive + approves to restrict approval of the PRR to your team/alias.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That OWNERS file will be created by the KEP author - can we somehow enforce that it will be added and will have the no-parent stuff?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can definitely add something as a presubmit to enforce this!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd agree with @spiffxp that no_parent_owners is favorable over creating a new workflow here.

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 17, 2020
@johnbelamaric
Copy link
Member Author

I updated this based on @wojtek-t 's changes to the PRR questionnaire. I think we can merge this as-is, with these follow ups needed:

  1. Remove the production-readiness.md from the community repo.
  2. Add a prr-review team (in OWNERS_ALIASES)
  3. Add KEP tooling to verify a PRR reviewer is present on a KEP

a cloud provider API, or upon an external software-defined storage or network
control plane.

For each of the fill in the following, thinking both about running user workloads
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

missing noun?

@neolit123
Copy link
Member

neolit123 commented May 14, 2020

this seems like an overall improvement for gating features with respect to quality control, so thank you @johnbelamaric and the PRR project for adding this to the template. i did early reviews of this work but lost the trail later.

one minor complain that i have, after trying to fill the answers in a WIP KEP is that the majority of the questions are not really applicable to e.g. a new {kubectl|kubeadm} ... command that performs a one time operation instead of persisting a feature that affects cluster behavior.

i guess in such cases there must be some sort a default answer, so i just went with "Not applicable" in a few places, but still trying to clarify "why".

something else is that possibly some of the details next to questions could have been commented out <!-- ... --> so that the questionnaire focuses on the questions and available options. once we start publishing KEPs to a website some of those details will be hidden, similarly to the rest of the template.

thanks.

For each of them fill in the following information by copying the below template:
- [Failure mode brief description]
- Detection: How can it be detected via metrics? Stated another way:
how can an operator troubleshoot without loogging into a master or worker node?
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo in loogging

see-also:
- "/keps/sig-aaa/1234-we-heard-you-like-keps"
- "/keps/sig-bbb/2345-everyone-gets-a-kep"
replaces:
- "/keps/sig-ccc/3456-replaced-kep"

# The target maturity stage in the current dev cycle for this KEP.
stage: alpha|beta|stable
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there is no such field in the Proposals structure, causing kepval / make verify to fail:
https://github.com/kubernetes/enhancements/blob/master/pkg/kepval/keps/proposals.go#L49-L76

looks like the template itself is not validated.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this has since been fixed, thanks

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/enhancements Issues or PRs related to the Enhancements subproject cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/kep Categorizes KEP tracking issues and PRs modifying the KEP directory lgtm "Looks good to me", indicates that a PR is ready to be merged. sig/architecture Categorizes an issue or PR as relevant to SIG Architecture. sig/release Categorizes an issue or PR as relevant to SIG Release. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
Enhancements Subproject
  
PRs - Blocked, or Author Inactivity i...
Development

Successfully merging this pull request may close these issues.

None yet