New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Propose SIG-Platform-Dev charter. #2033

Closed
wants to merge 5 commits into
base: master
from

Conversation

@enisoc
Member

enisoc commented Apr 11, 2018

This is the proposed charter for SIG-Platform-Dev, to be reviewed by the steering committee.

The important files in this PR are:

  • sigs.yaml
  • sig-platform-dev/charter.md

All other files/changes were produced automatically by make generate.

@parispittman

This comment has been minimized.

Contributor

parispittman commented Apr 11, 2018

/committee steering

@timothysc

This comment has been minimized.

Member

timothysc commented Apr 12, 2018

/assign @jbeda - because I know he has thoughts here.

@munnerz

This comment has been minimized.

Member

munnerz commented Apr 12, 2018

/cc

@k8s-ci-robot k8s-ci-robot requested a review from munnerz Apr 12, 2018

- enisoc
- jzelinskie
- enisoc
- pwittrock

This comment has been minimized.

@sttts

sttts Apr 13, 2018

Contributor

As an owner of 4 and initiator of 3 of the 6 subprojects and as a link to Api-Machinery, I officially volunteer to become a lead in this group: enisoc#1

This comment has been minimized.

@enisoc

enisoc Apr 13, 2018

Member

I am enthusiastically in favor. Stefan has been working for a long time on making extension APIs more approachable to developers who don't work on core k8s by working on samples and generators.

@jzelinskie @pwittrock Let me know if you have any objections to me merging enisoc#1.

This comment has been minimized.

@jzelinskie

jzelinskie Apr 13, 2018

I'm absolutely in favor of this.

This comment has been minimized.

@sttts

sttts Apr 16, 2018

Contributor

Thanks :-)

- **gengo**
- Owners:
- https://raw.githubusercontent.com/kubernetes/gengo/master/OWNERS
- **code-generator**

This comment has been minimized.

@sttts

sttts Apr 13, 2018

Contributor

what is the rational to have gengo+code-generator here? This looks like foundational tech that is deeply linked to API machinery.

This comment has been minimized.

@enisoc

enisoc Apr 13, 2018

Member

You're probably right since you know these tools much better than I do. For reference, my initial rationale was to pull out things that "generate concrete code that calls into abstract SIG-AM libraries", as opposed to the abstract libraries themselves that SIG-AM writes by hand.

For example: SIG-AM defines abstract interfaces with methods like DeepCopy(), DeepCopyInto(), DeepCopyObject(), etc. They don't really care how you choose to implement those interfaces, just that you conform to them.

It makes sense to me for SIG-PD to own a tool like deep-copy generator that gives you the option of implementing those interfaces through code generation. You could also choose to implement DeepCopy() based on reflection, or hand-write all your copies, or use a different generator.

Similarly, I envision SIG-AM owning client-go libraries like rest and cache, which implement abstract behavior for talking to API servers.

It makes sense to me for SIG-PD to own a tool like client-gen that gives you the option of wrapping these abstract libraries in generated, statically-typed code. Neither client-go nor the API server cares if you use this generator though. They are perfectly happy if you use the dynamic client, or manually instantiate a RESTClient and your own Reflector, Indexer, and Informer as needed, rather than generate a statically-typed factory for these things.

This comment has been minimized.

@pwittrock

pwittrock Apr 13, 2018

Member

I am not sure that I agree these tools should be subprojects of the SIG. The test-framework will still be owned by SIG-testing, the docs generation will still be owned by SIG-docs. The goal of the SIG is to integrate components owned by other SIGs into user friendly toolkit. IMHO most of components it is integrating shouldn't be owned by the SIG unless that is good cause.

This comment has been minimized.

@enisoc

enisoc Apr 13, 2018

Member

Integrating tools owned by multiple SIGs is a good way of putting it. For example, kubebuilder integrates both client generation (clients are in scope for SIG-AM) and controller generation (controllers are out of scope for SIG-AM).

I'm not opposed to leaving these in SIG-AM if there is consensus that it makes sense.

This comment has been minimized.

@pwittrock

pwittrock Apr 13, 2018

Member

I guess one question is how tightly coupled are these to the apimachinery libraries. IIRC they are relatively tightly coupled. If that is the case, I think the ownership should stay together.

This comment has been minimized.

@sttts

sttts Apr 16, 2018

Contributor

Their output is tightly coupled with api machinery, i.e. the output is useless (maybe with the exception of deepcopy) outside of api machinery.

I also like the term "Integrating tools" as a rule of thumb for Sig-PD.

This comment has been minimized.

@pwittrock

pwittrock Apr 18, 2018

Member

If the output is tightly coupled with api machinery I would expect the output of the tools to be owned by api machinery. @sttts thoughts?

This comment has been minimized.

@sttts

sttts Apr 19, 2018

Contributor
  • output => AM
  • user experience invoking them => PD
  • semantics of the code tags => part of user experience => PD
- Seed members established at subproject founding
- *MUST* be an escalation point for technical discussions and decisions in the subproject
- *MUST* set milestone priorities or delegate this responsibility
- *MUST* remain active in the role and are automatically removed from the position if they are unresponsive

This comment has been minimized.

@smarterclayton

smarterclayton Apr 13, 2018

Contributor

This seems like we should clarify unresponsive to exclude preplanned leave that the other sig leads are made aware of and some due dilligence to have responsibilities shouldered. Ie maternity leave is not grounds for removal, bad faith radio silence is.

This comment has been minimized.

@enisoc

enisoc Apr 13, 2018

Member

This is copied verbatim from the template. Can we fix it there and then copy it out to all SIG charter drafts?

This comment has been minimized.

@derekwaynecarr

derekwaynecarr Apr 14, 2018

Member

I have language that reflects @smarterclayton sentiment in the sig-node charter I am writing with dawn. I think it should be added in the global template and wanted to bring it up in steering committee for discussion in next meeting.

This comment has been minimized.

@pwittrock

pwittrock Apr 18, 2018

Member

FWIW, I have a longer template out that does a better job with this, but was considered too verbose by a number of of folks. I'll work on getting the longer template merged and update the short template with something.

Link:
https://github.com/kubernetes/community/pull/1650/files#diff-16973d738c2229e3dac80e77e149200eR68

This comment has been minimized.

@enisoc

enisoc Apr 18, 2018

Member

Instead of copying it into each SIG charter, can we post a single copy somewhere and link to it from each SIG charter? Not for the whole template, but for things like these membership rules that we expect few SIGS to customize.

## Subprojects
The following subprojects are owned by sig-platform-dev:

This comment has been minimized.

@pwittrock

pwittrock Apr 13, 2018

Member

kubebuilder?

This comment has been minimized.

@enisoc

enisoc Apr 13, 2018

Member

I lumped kubebuilder in with the "experimental subprojects" section in the charter because there were claims on the original thread that it and metacontroller are out of scope for the Kubernetes project. I disagree on both counts, but in light of the concerns raised, I decided to decouple that decision from the founding of SIG-PD and leave it up to the KEP process to decide later.

This comment has been minimized.

@pwittrock

pwittrock Apr 13, 2018

Member

That is fine.

This comment has been minimized.

@pwittrock

pwittrock Apr 14, 2018

Member

That is fine for now.

- **pdk-roadmap**
- Owners:
- https://raw.githubusercontent.com/kubernetes-sigs/pdk-roadmap/master/OWNERS
- **sample-apiserver**

This comment has been minimized.

@pwittrock

pwittrock Apr 13, 2018

Member

What is our intention of owning these as subprojects. The goal of the SIG is to build a toolkit for API extensions. Would the samples be rewritten on top of this toolkit?

This comment has been minimized.

@enisoc

enisoc Apr 13, 2018

Member

I see the existing samples as a proto-PDK -- an early attempt to start a PDK within SIG-AM that didn't get very far yet because SIG-AM has many other components of its mission that took priority. By contrast, producing samples for use by non-core devs is a primary part of SIG-PD's mission, so I think we should own what exists now and push it towards a cohesive vision.

This comment has been minimized.

@munnerz

munnerz Apr 13, 2018

Member

👍 whilst I think demonstrating a "minimal" example is useful, ultimately best practice is the end-goal (and always has been).

That said, I don't think a simple switch to apiserver-builder right now is sufficient. Unless I misunderstand, apiserver-builder will not allow deep customisation of behaviour of a generated controller. This should be solved within the apiserver-builder repo before promotion to sample-*.

The general trend of moving components into apiextensions only emphasises this point imo. Perhaps dog-fooding is our bench mark - if we can write apiextensions-apiserver with apiserver-builder, maybe we are ready to recommend it as a best approach and switch more examples to it 😄

That said, I am generally in favour of moving sample-* under the sig-pd charter in order to meet these goals.

This comment has been minimized.

@sttts

sttts Apr 16, 2018

Contributor

Next to being an example for 3rdparties to copy, sample-apiserver is also a testbed of using k8s.io/apiserver, i.e. it is a proof that the library can be used externally. Porting it to apiserver-builder neglects this 2nd purpose.

This comment has been minimized.

@pwittrock

pwittrock Apr 17, 2018

Member

@munnerz Thanks for the reply. I wasn't suggesting apiserver-builder, but was more asking about what our goals with the samples are. Do we want to keep these around as examples of how to use the apimachinery libraries directly? If so, is SIG PD the correct owner or are the owners of the libraries themselves the correct owners? If the goal of the SIG is to develop a toolkit is it ok to delete the samples and replace them with documentation on how to use the toolkit once it exists?

This comment has been minimized.

@sttts

sttts Apr 17, 2018

Contributor

is it ok to delete the samples and replace them with documentation

No, compare my comment. Potentially we can move or rename them some day.

replace them with documentation on how to use the toolkit once it exists?

It's early and not of much use to discuss those details before we have something like a toolkit or at least knowing how it will look like. I am not sold on the idea that we need a heavy layer on-top of k8s.io/apiserver. IMO we should make k8s.io/apiserver easy to use for the common use-case.

If so, is SIG PD the correct owner

Enabling k8s.io/{apiserver+code-generator} for easy consumption is in scope of SIG PD for sure. SIG PD should own (and push to improve, actively with PRs, not only begging SIG AM for changes) the provided 3rdparty user-experience of the foundational libraries. Hence, owning the samples makes a lot of sense.

This comment has been minimized.

@pwittrock

pwittrock Apr 18, 2018

Member

@sttts

Next to being an example for 3rdparties to copy, sample-apiserver is also a testbed of using k8s.io/apiserver, i.e. it is a proof that the library can be used externally. Porting it to apiserver-builder neglects this 2nd purpose.

This seems like an argument for these to be owned by api machinery SIG. SIGs should own their own tests.

Enabling k8s.io/{apiserver+code-generator} for easy consumption is in scope of SIG PD for sure.

There are a number of ways to enable easy consumption.

SIG PD should own (and push to improve, actively with PRs, not only begging SIG AM for changes) the provided 3rdparty user-experience of the foundational libraries.

I'd loved to see the upstream code-generators UX polished, and even submitted a proposal for doing so: #1315. That said, because the stability of the code generators is important, it may be simpler and faster to develop UX that wraps the upstream libraries - e.g. what kubebuilder and apiserver-builder do.

- **gengo**
- Owners:
- https://raw.githubusercontent.com/kubernetes/gengo/master/OWNERS
- **code-generator**

This comment has been minimized.

@munnerz

munnerz Apr 13, 2018

Member

Where does kube-openapi fall within this charter?

As much as it tries to be a general purpose framework/library, it is closely tied to gengo and what with the proliferation of API extensions, will become more important as time goes on (wrt automatic generation of api validation schemas, or docs). Would this fall within sig-pd?

This comment has been minimized.

@sttts

sttts Apr 16, 2018

Contributor

I think it is in the same class as k8s.io/code-generator: a foundational tool deeply linked to k8s.io/apiserver.

I would use the term "integrational tools" vs. "foundational tools" as the rule of thumb between PD and AM. This clarification does not mean that we cannot improve the user-experience of these tools and to broaden its purpose for 3rdparty developers.

sttts and others added some commits Apr 13, 2018

Adding myself as chair
Following offline discussion with @enisoc, help with facilitating meeting
is welcome. I am very happy to support with this.
@k8s-ci-robot

This comment has been minimized.

Contributor

k8s-ci-robot commented Apr 16, 2018

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
To fully approve this pull request, please assign additional approvers.
We suggest the following additional approver: jbeda

Assign the PR to them by writing /assign @jbeda in a comment when ready.

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@jbeda

This comment has been minimized.

Contributor

jbeda commented Apr 18, 2018

Sorry I didn't wade in on this until now -- was on vacation last week.

My biggest question: why isn't client-go part of this SIG? That is the foundational piece of any SDK/PDK that we build. It is what all of the other potential sub projects are based on.

(Similarly, I think the rest of the language client libraries should be here too.)

If our goal is to get to the point where we have loose coupling between control plane components and we have flexibility around what makes up the control plane, then we need to have loose coupling for client-go. It will be painful but, IMO, is necessary to remove the "gatekeeper" nature of the k8s control plane as it exists today. (Having a loosely coupled client-go, for instance, is going to be critical for things like breaking out the cloud providers.)

If we don't include client-go then I'm not convinced that there is critical mass enough to support a SIG. The stuff that is left is ancillary and doesn't make up a PDK in any real way.

@enisoc

This comment has been minimized.

Member

enisoc commented Apr 18, 2018

@jbeda wrote:

My biggest question: why isn't client-go part of this SIG?

The out of scope section of the charter is our answer to this question. Can you point out the parts of that argument that you disagree with?

If we don't include client-go then I'm not convinced that there is critical mass enough to support a SIG. The stuff that is left is ancillary and doesn't make up a PDK in any real way.

The scope section of the charter lays out the mass of work we think needs to be done.

We agree that all this work is ancillary to the Kubernetes project, in the sense that it "provides necessary support to the primary activities or operation of an organization or system".

Maintaining the Kubernetes API server and its first-party clients is a "primary activity" and lives in SIG-AM. Creating a development kit to help external consumers use those things (as well as controller machinery from sig-apps) is ancillary "support activity" that lives in SIG-PD, just like the Windows Driver Kit is ancillary to the Windows OS/Kernel.

* A framework (generators and libraries) for building extension APIs and controllers.
* [metacontroller](https://github.com/GoogleCloudPlatform/metacontroller)
* A hook-based wrapper to simplify adoption of core controller code, patterns, and best practices.

This comment has been minimized.

@bassam

bassam May 11, 2018

could we please add rook's operator-kit to this list. https://github.com/rook/operator-kit

@idvoretskyi

This comment has been minimized.

Member

idvoretskyi commented May 15, 2018

/cc

@k8s-ci-robot k8s-ci-robot requested a review from idvoretskyi May 15, 2018

@k8s-ci-robot

This comment has been minimized.

Contributor

k8s-ci-robot commented May 29, 2018

@enisoc: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@enisoc

This comment has been minimized.

Member

enisoc commented Jun 20, 2018

I'm closing this PR because we've decided to start out by working on these issues within SIG API Machinery, at the recommendation of the steering committee. Please join the SIG-AM mailing list and meetings if you'd like to keep up with these efforts or contribute.

The new controller-oriented arm of SIG-AM has already begun to collaborate on several repos under the kubernetes-sigs org:

Work has also begun on a cohesive set of docs specifically aimed at controller authors:

http://book.kubebuilder.io/

@enisoc enisoc closed this Jun 20, 2018

@sttts

This comment has been minimized.

Contributor

sttts commented Jun 21, 2018

I'm closing this PR because we've decided to start out by working on these issues within SIG API Machinery, at the recommendation of the steering committee. Please join the SIG-AM mailing list and meetings if you'd like to keep up with these efforts or contribute.

I still think this is not going to work. We had hardly AM meetings in the past where we had those discussions in the necessary depths. IMO this can have two reasons:

  • either all the work and disucssions are under the radar of the SIG-AM meetings
  • or there is not much to talk about.

From discussion with 3rdparty developer about their pain using our libraries, the development of operator-framework+kubebuilder and the creation of those controler-runtime/tools repos suggests that there is more demand than is visible in AM.

Also I have never seen platform-dev topics on any AM roadmap.

So I am pretty sure it's the first of the two reasons. Do we agree that we keep it like that, i.e. under the radar? Or do we want to establish something (a working group? subgroup or whatever it is called) in addition?

@enisoc

This comment has been minimized.

Member

enisoc commented Jun 21, 2018

I still think this is not going to work.

Unfortunately, we couldn't convince the steering committee of that. So we're going to take their recommendation in good faith and do our best to build what we think our users need within the existing SIGs.

So I am pretty sure it's the first of the two reasons. ["all the work and discussions are under the radar of the SIG-AM meetings"]

Agreed. Before this, we lacked consensus on where such topics should be discussed. Those of us working on controllers thought that SIG-AM was focused only on the interface aspects (IDL, REST client/server), and not on the implementation behind such interfaces (in the form of level-triggered, reconciling control loops).

The steering committee recommended expanding SIG API Machinery to encompass both, and I have proposed writing this into SIG-AM's future charter explicitly to clarify its new scope. The current mission statement doesn't mention anything about control loops.

Do we agree that we keep it like that, i.e. under the radar? Or do we want to establish something (a working group? subgroup or whatever it is called) in addition?

We plan to increase visibility of controller topics in SIG-AM by showing up to SIG meetings, giving updates on our subprojects, using the SIG mailing list for discussion, and so on. I propose that we see how that goes for a bit and then discuss at a future SIG-AM meeting whether we need more structure.

@p0lyn0mial

This comment has been minimized.

Contributor

p0lyn0mial commented Jun 22, 2018

Hi there, I just want to express my sadness as I have been waiting for this SIG to eventually form. Not only because I agree with the others that this was good idea. But also because I wanted to offer my help and work with the others to make developers life easier especially the ones that intend to extend and consume Kubernetes APIs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment