KEP-1a: Meta KEP Implementation

[justaugustus]

This is a provisional meta-KEP, which discusses iterative improvements to the KEP process.

rel:

• Separating proposals and architecture related items from community #2565
• SIG Architecture meeting mentioning the need for a clearer KEP process - https://youtu.be/MMJ-zAR_GbI
• SIG Architecture meeting agreeing to move forward with a KEP process improvement effort to be co-led with SIG PM (@justaugustus / @jdumars) - https://youtu.be/fmlXkN4DJy0

/sig pm architecture
/assign @jdumars @calebamiles @idvoretskyi @timothysc

Signed-off-by: Stephen Augustus foo@agst.us

[justaugustus]

Adding Tim, Aish, and Aaron to weigh in on Release Team implications.

/assign @tpepper @AishSundar @spiffxp
/sig release

[justaugustus]

Adding an explicit hold until we gather some feedback:
/hold

[timothysc]

Just to be honest, this document was confusing up until this point.

[timothysc]

Also, this KEP will likely be dead after the transition... but whatevers.

[justaugustus]

@timothysc -- were there specific parts that were confusing? The motivation was "create a KEP that speaks about the improvements we need to make to the process, while ensuring (in the interim) that the process doesn't change underneath SIGs who have adopted it in its' current state".

And yep, this KEP will effectively die once we re-implement KEP-1.

[timothysc]

This might be a bit orthogonal imo. SIGs react on the fly.

[justaugustus]

Currently, SIGs react on the fly.
Ideally, moving forward, SIGs should be able to plan and provide some level of soft commit prior to the release cycle, and that information should be aggregatable through the metadata in each KEP. The shape of how exactly to aggregate the data hasn't been defined yet, though.

[timothysc]

Without specifics, I would remove.

[justaugustus]

@timothysc -- removed

[erictune]

Can you say more about how you expect end users to use the new repository structure?

If I am an end user who wants to find out about the design of Kubernetes, what URL should I start at? For Python, there is one page, called https://www.python.org/dev/peps/ that lists all their PEPs (~=KEPs).

[justaugustus]

End users should be able to hit https://keps.k8s.io, which would be a redesigned version of https://contributor.kubernetes.io/keps/.

I envision this site / repo having at least three directories:

• keps/ (KEPs)
• design-proposals/ (historical design proposals from https://git.k8s.io/community/contributors/design-proposals)
• arch[itecture]|design/ (design principles of Kubernetes, derived from reorganizing https://git.k8s.io/community/contributors/devel, mentioned here)

[erictune]

Note that Python solves this, AFAICT, by having a single PEP, called PEP 0, which is an index of all the PEPs. You reserve a number by sending a commit to add a row to the table in PEP 0.

[justaugustus]

I feel that having a KEP index is a good idea, but sending a commit to add a row still leaves the race condition that we had with who chooses a number first.

I'd imagine that an index would simply be generated by some script we use to walk the repo and then committed alongside KEP changes e.g., make generate for sig-list.md

[erictune]

So, then KEPs don't have a number until they are accepted? This makes talking about them harder? Also, don't we want to keep a record of non-accepted KEPs as a record of what didn't work? Python allocates numbers to all proposals and tracks rejected and withdrawn as well as accepted PEPs

[justaugustus]

Good point about the KEPs that aren't in some accepted state. I've updated the copy to say we'll track based on the GitHub issue number of the KEP tracking issue.

[justaugustus]

@timothysc @erictune -- PTAL again. Hoping that I've addressed your comments.

I've also added a proposed directory structure as well as the metadata structure that would be included per KEP directory.

[calebamiles]

/approve

[justaugustus]

@erictune -- apologies for not adding that. Just pushed a commit to reference the directory structure.
I'm in agreement with Caleb and Jaice's notes w.r.t. metadata and repo organization.

We'll be able to present in a way that's accessible, which may very well include an index. I really like that idea.

[justaugustus]

I'd like to thank everyone who's reviewed thus far. It's been extremely valuable to get your input, especially from the historical aspect! ❤️

Given that this KEP is currently marked as provisional, I'd like to keep in the spirit of merging early and iterating as we move through the KEP statuses:

Avoid getting hung up on specific details and instead aim to get the goal of the KEP merged quickly. The best way to do this is to just start with the "Overview" sections and fill out details incrementally in follow on PRs. View anything marked as a provisional as a working document and subject to change. Aim for single topic PRs to keep discussions focused. If you disagree with what is already in a document, open a new PR with suggested changes.

If you feel that what we've laid out is a good start and moving in the right direction, please drop a /lgtm on this when you have a chance. Thanks!

(As mentioned on this SIG Architecture thread, we'll be timeboxing the comment period for this to today, Wednesday, 9/19 at 6pm PT.)

[erictune]

@jdumars wrote:

the highest stakeholders which are subproject participants first and foremost, SIGs second, consumers like the release team third, and causal observers dead last.

I don't agree that casual observers (users if you will) are dead last.

@calebamiles wrote:

I do like that for [smaller] projects like Rust you can see all the content in one place
the internet (IETF) is a bigger project, and yet still manages to have a single page view of all proposals.

[jbeda]

Lots of good ideas here. Thank you for taking this up! My biggest concern is that this is a lot of changes and we'll have to write up good instructions for folks. I think the biggest mistake we've made up until now is introducing KEPs without tooling. We could also have done a better job of providing "how to" instructions so that folks working with KEPs understand clearly the workflow.

IMO, let's make sure we prioritize that before anything else.

[jbeda]

The original thinking here is that there would broadly be two types of keps -- those that are "local" to a SIG whereby the SIG is reusing the KEP process for their own internal decision making process. This might leak out where it is mostly in one SIG with a few other SIGs weighing in.

Contrast this to KEPs that are wide reaching and impact most of the project. These should get wider review and have a wider set of approvers. I can see situations where something might start out "SIG local" but become "k8s wide" as the implications are realized.

I don't think we did a good job of making this clear.

I think we should either make it clear that there are two types of KEPs and perhaps have another top level directory for sig-local keps (sig-keps?) or we should just say that every kep is top level and collapse the hierarchy.

That being said -- I'd suggest that we move forward in some concrete ways that will improve things. We can always change this again in the future if we need to.

[jbeda]

I do like the idea of having a directory for keps. There are then options to make it bigger and evolve it over time. I would look to minimize the amount of required stuff here and not be too prescriptive of the breakdown. We already have a problem where there are a lot of rules and it is hard to get people to understand the purpose of them.

[jbeda]

I'm not a huge fan of this but I guess it is ok. We'll have sparse numbering as there will be other PRs.

One thing that I want to do regardless is encourage people to check in early. If the latency between PR for new kep and commit is small these races will be minimal. We can and should build some presubmit to prevent races too.

In any case, if we use the PR number as the KEP number there will still have to be a dance where people will create a change, submit a PR, update their files and submit/push another change to that PR. That dance isn't obvious and will make this a bit harder.

[jbeda]

We are already building out contributors.k8s.io and we should put the keps there. That was one of the main reasons for that site. /cc @castrojo

[castrojo]

We can totally do this, we just need the repo URL when it's created and we can implement it. Doing it with three sections wouldn't be a problem. Hugo can be sensitive to the metadata though, it'd be great if the tooling could lint it before hugo consumes it.

[jbeda]

@erictune:

I dislike that https://contributor.kubernetes.io/keps/ is organized alphabetically

I agree -- the number should out in front. I think this is just a point in time issue. @castrojo -- Where should Eric file an issue.

[jbeda]

Also -- merge early! Let me know when you want an LGTM label and I'll happily put it on there. Don't want to do it now to prevent a surprise merge.

[jdumars]

@erictune sorry I wasn't clear enough on my comment. I meant casual observers as people who do not have any stake in either consuming or generating the information. Many people "weigh in" on things as an academic exercise. I'd prefer that this site be focused on meeting user and generator needs.

[castrojo]

@erictune and others, https://github.com/kubernetes-sigs/contributor-site/issues is where site issues should be. I am filing other issues for the design proposals and architecture designs as well. Please feel free to file issues and recommendations. Happy to share OWNERship with anyone who wants to rev faster/contribute.

[justaugustus]

@jbeda -- thanks for the feedback. This is ready for an /lgtm and merge!
/hold cancel

[justaugustus]

FYI: I've created a KEP Implementation Tracking project board, which captures some of the feedback here.

[jbeda]

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: calebamiles, jbeda

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• keps/OWNERS [calebamiles,jbeda]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[jbeda]

/lgtm

[justaugustus]

/remove-sig pm
/sig architecture
/area enhancements