"v2" API (API/client redesign umbrella issue)

[bgrant0607]

It's clear we're not going to be able to make all the changes we'd like in the v1 API (#7018), so this issue is to track/brainstorm things we might want in the v2 API.

Last updated 6/19/2015

API/client overhaul:

• Top-level API object plugins: API plugin design thread #991, Umbrella: Allow a 3rd party plugin to run on top of a Kubernetes cluster #3201, Break up types.go #1744, RFC: all controllers are separate web services. #703, Shared Pod Storage (e.g. Config Storage) #6923
  • Convert ReplicationController to a plugin Convert ReplicationController to a plugin (was ReplicationController redesign) #3058, rename it to ReplicaSet Create ReplicaSet #3024, and recursively delete pods by default (see graceful termination Consistently support graceful and immediate termination for all objects #1535, below)
  • Daemon Daemon (was Feature: run-on-every-node scheduling/replication (aka per-node controller or daemon controller)) #1518
  • Job Job Controller #1624
  • Deployment Deployment resource (was Evaluate adoption of OpenShift template configs and/or deployments) #1743
  • ConfigData Config Resource Proposal #6477, Support dynamic configuration distribution #1553, Store master and node config in API #1627, Integrate third-party configuration management tools with our config distribution API #2068, Remove configuration flags #5582
  • Rename PersistentVolumeClaim to PersistentVolume, PersistentVolume to StorageVolume (or similar) as per comment
• Subobject plugins: discussed in Support OCI images in API #7203 and lots of other places
  • I'd also like to consider replacing annotations with a list of plugins Improve Annotation Format #12226
  • volume sources should be converted to plugins
• Multiple API prefixes/groups, including an experimental API prefix: split api in to cluster and pod/user portions #3806, Communication between Kubernetes components #6363, Supporting multiple API prefixes #10009, How to structure independent api endpoints in kubernetes #635, GetReference shouldn't hardcode api prefix #1490
  • API group in addition to kind and apiVersion: Proposal: Long term evolution of runtime.Scheme / conversion.Scheme #7111, Supporting multiple API prefixes #10009, [RFC] adding a proposal for api groups #12951
  • Modular API: Break up types.go #1744, Add a bastion to apiserver (reverse lookup by IP address + proxy) #3481 (API subsets), Bulk creation API #9633 (comment) (separate client objects from API objects)
  • Provider/domain field (e.g., kubernetes vs. docker) -- https://github.com/projectatomic/atomicapp
  • Some hypothetical API groups:
    • end-user execution objects: pod, pod template, secret, config data, persistent volume claim
    • end-user networking-related objects: service, ingress
    • end-user app deployment objects: replica set, deployment
    • end-user batch objects: job, scheduled job
    • app admin objects: namespace, limit range, resource quota
    • identity/auth objects: service account, user (future)
    • system / tooling objects: event, endpoints, component status, resource metrics
    • infrastructure / cluster admin deployment objects: daemon set
    • infrastructure / cluster admin infrastructure objects: node, persistent volume
    • experimental apis and plugins
• Binary encoding: Investigate alternative JSON parsers #3338, Protobuf communications #8132 (proto)
• Split objects in etcd to mitigate false conflicts: API: frequent ResourceVersion changes (from status updates) cause false conflicts #8625
• API support for apply: v2 API proposal "desired vs actual" #17333
• Restructuring: apiserver: re-architect storage #1451, Our API code is hard to understand #2306, Break genericapiserver and API machinery into separate repos #2742, Add more structure to our directories, and document it #4851, Make it easy to include a kubernetes client library in a separate Go project #5660, Defaults and Conversions have deep entanglement #6210, Proposal: Long term evolution of runtime.Scheme / conversion.Scheme #7111, Refactor kubectl to decouple command framework from business logic #7311, Fuzzer should be aware of API defaults #7337, Versioned API objects should be in a self-contained package #7584, Remove json tags from pkg/api/types.go #3933
• Client versioning: Eliminate round-trip conversion of API objects in kubectl convert #3955, Versioned clients #4874
• Fork versioned client on every breaking change
• Versioned validation: Move validation to be per-API-version #8550, Field names in validation errors need to be translated from internal name to external name #3084
• Strict validation: Stricter server-side validation of JSON entities #5889
• Tag-driven, swagger-documented validation: Add validation constraints to API struct tags #8116
• Fine-grain versioning: Cluster versioning #4855, Supply server capabilities along with version #2953
• Return more info on failure: kubectl should return more information on failure #6132
• Fix syntax for field validation errors: https://github.com/kubernetes/kubernetes/pull/12035/files#r36374661
• API test fixtures: Test fixtures to generate valid objects for testing #5121, Validation tests are a mess #4894
• Cleanup object references and selfLink: Deodorize ObjectReference and SelfLink #6596, Don't store selfLinks #2797, GetReference shouldn't hardcode api prefix #1490, Need validation for ObjectReferences #4651, Centralize api resource path construction logic #4549
• Kubelet to use apiserver: Clean up Kubelet RESTful APIs #2098
• REST best practices and HTTP correctness: Review API for application of best practices #4187, Something calls /version with incorrect user agent "Go 1.1 package http" #5672, Api server proxy does not handle redirect headers #6387, Complete atomic update code for all objects and create #656
• Swagger correctness: Some struct fields missing in swagger-spec #6842, Document API return values in swagger #6167, Use dateTime type in swagger #2968, Document custom JSON types in swagger #2971, Swagger represents all maps as type "any" #4700, Validate URL parameters #3714
• Swagger 2.0: Add support for Swagger 2.0 emicklei/go-restful#157
• Form and client generation: Facilitate automatic form and client generation #6487, add Description and validation properties to EnvVar in the REST API so that environment variables can have better tooling #4210, Compatibility with GCP deployment manager #3685
• Improved API documentation: Automatic Generation of Swagger Documentation in Types #10933, Attach documentation to API types and/or resources #5331
• Change all fields to pointers for uniform detection of unspecified fields
• Scale to 1000 nodes / 50k pods

API feature changes:

• Finish subresource work, such as Properly set object status #2726, Create subresource end points for replication controllers #4909, Support for /watch in api paths should be removed #8337, Delete /binding #10043
• Bulk API: Bulk creation API #9633, Bulk event creation #3279
• Initializers: Places for hooks #3585
• Dependencies and graceful termination for all objects: Consistently support graceful and immediate termination for all objects #1535
• Per-object generation numbers for all objects: Create per-object sequence number and report last value seen in status of each object #7328
• Generalized label selector: Generalize label selectors #341
• Generalized field selectors: Generic field selectors #1362, Filtering resources by structured data (e.g., annotations, fields) #4476
• Reverse label lookup: Reverse lookup by labels #1348
• Get by uid: Efficient lookup by label selection, reverse label selection, uid, and IP addresses #4817
• Get subset of fields: GET/LIST subset of object fields #1459, Make API list operations consistent #2740
• Generalized image representation: Generalize container image representation #8702
  • ACI images: Support OCI images in API #7203
• Eliminate most volume sources
• Eliminate gitRepo in favor of a generic container volume mechanism Image volumes and container volumes #831
• Eliminate phase Pod Phase - Vacating.  #6951
• A usable mechanism for waiting for conditions Allow users to wait for conditions from kubectl and using the API #1899
• Replace object names with specialized object references: Create the v1 API #7018 (comment), Refactor references to object by Name to LocalReference #8503
• Links to related objects in status Links to related object in json responses #3676
• Watch improvements: Create the v1 API #7018 (comment), Create the v1 API #7018 (comment), List/Watch events for objects matching a label query #3295, Feature Request: Ability to WATCH more than one resource type #1685, Single watch endpoint for events and entities changes #5130
• Endpoints: We'll need a way to avoid quadratic behavior (one endpoint changing returns whole set).
• Final resolution on minion Replace use of term "Minion" #1111
• More service changes Service reorg ideas #2585
• Generalize deletion/mutation prevention: Generalize prevention of accidental deletion / mutation #10179

[derekwaynecarr]

Mechanism to set initializers / places for hooks prior to scheduling #3585

[lavalamp]

Replace ServiceAccountSecret secret type with a cluster account secret type. Secret types should be named for their contents, not their sources, IMO. (source can live in an annotation or something)

[bgrant0607]

@lavalamp Is there a reason we can't fix ServiceAccountSecret now? It was just merged recently.

[pmorie]

Agree, why not just fix it now?

On Thu, May 14, 2015 at 1:25 AM, Brian Grant notifications@github.com
wrote:

@lavalamp https://github.com/lavalamp Is there a reason we can't fix
ServiceAccountSecret now? It was just merged recently.

—
Reply to this email directly or view it on GitHub
#8190 (comment)
.

[davidopp]

/subscribe

[lavalamp]

@bgrant0607 I'm OK with changing it now, but lack cycles to do it myself.

[lavalamp]

@liggitt... wrong issue? or maybe you can explain in more detail?

[liggitt]

@lavalamp my bad, I misread "cluster account secret type" as "cluster-scoped"

[bgrant0607]

Issue filed for ServiceAccount secrets: #8271.

[bgrant0607]

Punted from v1:

• Eliminate most volume sources
• Eliminate phase Pod Phase - Vacating.  #6951
• Replace object names with specialized object references: Create the v1 API #7018 (comment)

[bgrant0607]

Other use cases for subobject plugins:

• Secret providers, such as dockercfg
• Cloud-provider-specific load-balancing info communicated from service controller to kube-proxy (or plugin)

[bgrant0607]

LocalReference: #8503

[eparis]

/subscribe @eparis

[erictune]

@bgrant0607 I assume you are just looking to collect breaking changes here, and not few features?

[erictune]

Suggest we change pointer-to-scalar to pointer-to-struct everywhere, for possible future compat with proto.

[bgrant0607]

@erictune Yes, you are correct.

[bgrant0607]

Generalized image representation: #8702

[bgrant0607]

In addition to eliminating volume sources obsoleted by persistent volume claims (gcePersistentDisk, awsElasticBlockStore, nfs, iscsi, glusterfs, rbd), I'd also like to eliminate gitRepo in favor of a generic container volume mechanism (#831).

[pmorie]

+100 to container volume
On Wed, May 27, 2015 at 3:20 AM Brian Grant notifications@github.com
wrote:

In addition to eliminating volume sources obsoleted by persistent volume
claims (gcePersistentDisk, awsElasticBlockStore, nfs, iscsi, glusterfs,
rbd), I'd also like to eliminate gitRepo in favor of a generic container
volume mechanism (#831
#831).

—
Reply to this email directly or view it on GitHub
#8190 (comment)
.

[lavalamp]

I'd also like to eliminate gitRepo in favor of a generic container volume mechanism

+1

[lavalamp]

@bgrant0607 I have been thinking for a while now that it's bad that we have gcePersistentDisk and awsElasticBlockStore as volume types that pods need to specify. It seems to me that if we want true workload mobility, we need to abstract this into a separate type, so that you can use the same pod descriptions on any cloud provider. IOW pods shouldn't depend on on the implementation of persistence, they should just request a persistent volume, and (cluster admin via config? cloud provider?) determines the implementation that pods get.

[bgrant0607]

@lavalamp That's what persistentVolumeClaim is for. We just haven't removed the old fields yet.

[thockin]

Maybe it's just me, but there's a low-level annoyance I experience every time I try to look at API files - we document our API fields in terms of javascript-style names (leading lower-case) but Go demands we use leading upper-case. This makes for inconsistent comments and docs. Worse, Go will ignore case for JSON keys if it needs to, so there are probably lurking issues.

So my proposal is that if/when we start the v2 process, we consider explicitly using Go-style field names (leading upper-case). Other APIs (e.g. Docker) do it, and it's not so bad.

As a nice side-effect, go vet will stop complaining at us for a bunch of things.

[thockin]

Adding here so we don't forget (I hope).

Node.status.addresses uses 'type' as merge key but that is not sufficiently unique.

#73977

[thockin]

#81301

Probe.SuccessThreshold validation is contextual and ugly. "must be 1" in an API field is silly.

[bgrant0607]

Associative lists with non-standard keys (and even multi-field keys) were totally a mistake. We should have just used maps. Merging would have been so much simpler.

[alvaroaleman]

Associative lists with non-standard keys (and even multi-field keys) were totally a mistake. We should have just used maps. Merging would have been so much simpler.

@bgrant0607 So this is not true anymore? Or is it kept as a convention, but third parties that e.G. develop CRD-based APIs should consider using maps instead?

[wojtek-t]

As mentioned in #73527, we should use proper LabelSelector types for selector (instead of map[string]string as e.g. in Service or even string as in ListOptions) - they cost to much when being processed later many times.

[dprotaso]

related to #92226 it'd be nice to drop service links - or make them opt-in rather than opt-out.

[bgrant0607]

This issue should really be moved to a doc (or multiple docs, as not everything is API machinery) and updated.

[MadhavJivrajani]

/remove-kind design
/kind feature

kind/design will soon be removed from k/k in favor of kind/feature. Relevant discussion can be found here: kubernetes/community#5641

[sftim]

Given #8190 (comment), should we close this?

[bgrant0607]

I recommend closing. This issue is not actionable at this point.

[sftim]

/close

[k8s-ci-robot]

@sftim: Closing this issue.

In response to this:

/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[thockin]

If we ever do core/v2 (we should!!) this thread is a goldmine of ideas

…

On Tue, Dec 20, 2022 at 7:09 AM Kubernetes Prow Robot < ***@***.***> wrote: Closed #8190 <#8190> as completed. — Reply to this email directly, view it on GitHub <#8190 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABKWAVFMQBT2PSTLPIRWOMLWOHDZ7ANCNFSM4BDGARTA> . You are receiving this because you commented.Message ID: <kubernetes/kubernetes/issue/8190/issue_event/8079310130 ***@***.***>