groups/groups.yaml can become difficult to maintain

[neolit123]

i foresee that groups/groups.yaml will face the fate of k/test-infra/testgrid/config.yaml :) - i.e. one big yaml with a few owners that is difficult to edit and causes conflicts in PRs.

ideally groups should be sharded on the owner side.
can group settings be delegated on the side of group owners and perhaps only the group names can be enumerated in groups/groups.yaml ?

/kind feature
/priority important-longterm

[justaugustus]

Strong +1 for this!

[cblecker]

Maybe longterm, but right now the token to manage groups is extremely highly privileged.. I think only dims and I have access to it (the process is a manual run). So either way, we have to funnel approvals through a couple folks anyways. Sharding the config will only get us partially the way there.

[nikhita]

So either way, we have to funnel approvals through a couple folks anyways.

Can we go around this by setting up the sync as a postsubmit like we do for peribolos?

[cblecker]

Peribolos has a bunch of safeguards in place to prevent locking itself out from GitHub. We don't have those kinds of safeguards in place on the groups sync script.. right now, it requires a dry run, manual verification of the changes, and then a real live run.

If we had better testing and some of those safeguards in place, I'd be way more comfortable/confident with us doing this.

[thockin]

maybe we can break the monolith file into smaller files which are merged at run-time. At least it would be easier to read? Maybe even directory structure to separate RBAC groups from role groups from administrative groups

[fejta-bot]

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

[justaugustus]

/remove-lifecycle stale
/lifecycle frozen

[spiffxp]

#925 is a start at enforcing some conventions, though I'm not sure what "don't lock us out" tests we should have

[spiffxp]

I added #927 to address @cblecker's particular concern above

Longer term I agree sharding the config would be cool.

I personally think we need more conventions documented and enforced before we take that next step.

#928 should reduce some of the maintenance burden

[spiffxp]

Oh, one other note to address #460 (comment)

There are four humans who directly have the org admin role, in the event that the @kubernetes.io groups disappear or stop working

[nikhita]

Created #996

/assign

[spiffxp]

I recently updated slack-infra configs to shard out sig-testing resources. I really liked the approach of restrictions.yaml at root defining restrictions for what was allowed to be defined in sub-directories. I think modifying the groups reconciler to use the same approach would address my concerns about enforcing conventions/policies.

references:

• https://github.com/kubernetes/community/blob/master/communication/slack-config/restrictions.yaml
• https://github.com/kubernetes-sigs/slack-infra/blob/1eb97f5f2eb25163f26aba3e9ac6894b6a7c0133/tempelis/config/config.go#L33-L42
• https://github.com/kubernetes-sigs/slack-infra/blob/1eb97f5f2eb25163f26aba3e9ac6894b6a7c0133/tempelis/config/parser.go#L149

[spiffxp]

So I'm not sure whether to repurpose this issue or close it in favor of a new one. But to refresh on the definition of done....

I'm not going to allow approvers in groups/sig-foo/OWNERS until we have sufficient checks or enforcement in place to ensure that:

• someone can't quietly escalate privileges
• someone can't create random official-sounding groups without sufficient review/oversight

The former can probably handled by properly sharding groups, and enumerating which use cases don't need all the extra paranoia.

The latter could even be as simple as groups/whatever_test.go with a hardcoded list of groups that you've got to update every time you add a new group.

If someone wants to join the pool of approvers in groups/OWNERS, I'll allow it if they're going to work on implementing the above.

[justaugustus]

cc: @kubernetes/release-engineering, if this piques your interest.

@spiffxp -- Happy to jump in on groups/OWNERS to share the load.

It'd also be motivation to jump back into this refactor I was noodling on: uwu-tools/ggreconcile#1

[spiffxp]

Jumping in on OWNERS doesn't happen unless I see demonstrated commitment to implement the enforcement I described. A good demonstration would be PR's to improve the state of things, or authoring and driving consensus on a proposal and the necessary work items such that they meet the criteria of a help-wanted issue.

[nikhita]

A year too late for #996 (comment) .... created #2401

[spiffxp]

#2414 follows on from @nikhita's work

I'm realizing we have very, very little test coverage of how this thing actually works. I would feel more comfortable with more test coverage, and think it'd be a good help-wanted issue. But I don't think it's a complete blocker to proceeding with approver delegation.

I think what remains is:

• verify groups we definitely want root oversight over
• decide if there are any other policies to encode in tests
• start delegating approvers

[spiffxp]

/milestone v1.23

[spiffxp]

/remove-priority important-longterm
/priority important-soon
One other thought I had which maybe is just a nice to have: support loading in from arbitrarily named files. Same as we do for prowjobs. This allows contributors to organize however they see fit, instead of sticking to one massive file and sharding solely on approval boundaries.

I could see this being useful if, for example, we end up migrating mailing lists

[spiffxp]

/area groups

[spiffxp]

Looking to push this forward to done now that we have tooling able to support delegated approval. I've been going through and sharding out groups to sig-owned subdirectories, creating them as needed. What remains are the trickier corner cases, updating docs, and updating owners files.

Proposed path forward

• move groups to subdirectories: groups: shard some group definitions to subdirectories #2675
• move more groups to subdirectories: groups: shard some more group definitions to subdirectories #2683
• move remaining RBAC groups
  • k8s-infra-rbac-triageparty-cli: groups: more groups moves #2714
  • k8s-infra-rbac-sippy: groups: more groups moves #2714
• how to handle special case groups used for ACLs?
  • k8s-infra-artifact-security: since security@kubernetes.io is a member, make it PSC/SRC: groups: more groups moves #2714
  • k8s-infra-ii-coop: since working as part of wg-k8s-infra, make it k8s-infra: groups: more groups moves #2714
• how to handle special case staging groups for components bundled as part of kubernetes release? deferred to assign ownership of k8s-infra-staging-coredns and k8s-infra-staging-etcd #2726
  • k8s-infra-staging-coredns
  • k8s-infra-staging-etcd
• how to handle staging repos for artifacts produced by a wg?
  • k8s-infra-staging-multitenancy: inspected images, they relate to subprojects owed by sig-auth, so make it sig-auth: groups: more groups moves #2714
• how to handle staging repos for retired projects? deferred to decomission k8s-staging-bootkube #2725
  • bootkube
• how to handle staging repos for projects not part of kubernetes? deferred to: decomission k8s-staging-txtdirect #2727
  • txtdirect
• update docs to describe avoiding root groups.yaml: groups: more groups moves #2714
• update owners files in subdirectories to add approvers: groups: delegate approval to subdirectories #2715

[spiffxp]

/assign

[spiffxp]

/close
Followup issues opened for remaining groups in root

Announcement: https://groups.google.com/g/kubernetes-dev/c/DhPel8J8QiA

[k8s-ci-robot]

@spiffxp: Closing this issue.

In response to this:

/close
Followup issues opened for remaining groups in root

Announcement: https://groups.google.com/g/kubernetes-dev/c/DhPel8J8QiA

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.