Client package proposal

docs/proposals/client-package-structure.md

@@ -0,0 +1,349 @@
+<!-- BEGIN MUNGE: UNVERSIONED_WARNING -->
+
+<!-- BEGIN STRIP_FOR_RELEASE -->
+
+<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
+     width="25" height="25">
+<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
+     width="25" height="25">
+<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
+     width="25" height="25">
+<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
+     width="25" height="25">
+<img src="http://kubernetes.io/img/warning.png" alt="WARNING"
+     width="25" height="25">
+
+<h2>PLEASE NOTE: This document applies to the HEAD of the source tree</h2>
+
+If you are using a released version of Kubernetes, you should
+refer to the docs that go with that version.
+
+<strong>
+The latest release of this document can be found
+[here](http://releases.k8s.io/release-1.1/docs/proposals/client-package-structure.md).
+
+Documentation for other releases can be found at
+[releases.k8s.io](http://releases.k8s.io).
+</strong>
+--
+
+<!-- END STRIP_FOR_RELEASE -->
+
+<!-- END MUNGE: UNVERSIONED_WARNING -->
+
+<!-- BEGIN MUNGE: GENERATED_TOC -->
+
+- [Client: layering and package structure](#client-layering-and-package-structure)
+  - [Desired layers](#desired-layers)
+    - [Transport](#transport)
+    - [RESTClient/request.go](#restclientrequestgo)
+    - [Mux layer](#mux-layer)
+    - [High-level: Individual typed](#high-level-individual-typed)
+      - [High-level, typed: Discovery](#high-level-typed-discovery)
+    - [High-level: Dynamic](#high-level-dynamic)
+    - [High-level: Client Sets](#high-level-client-sets)
+  - [Package Structure](#package-structure)
+  - [Client Guarantees (and testing)](#client-guarantees-and-testing)
+
+<!-- END MUNGE: GENERATED_TOC -->
+
+# Client: layering and package structure
+
+## Desired layers
+
+### Transport
+
+The transport layer is concerned with round-tripping requests to an apiserver
+somewhere. It consumes a Config object with options appropriate for this.
+(That's most of the current client.Config structure.)
+
+Transport delivers an object that implements http's RoundTripper interface
+and/or can be used in place of http.DefaultTransport to route requests.
+
+Transport objects are safe for concurrent use, and are cached and reused by
+subsequent layers.
+
+Tentative name: "Transport".
+
+It's expected that the transport config will be general enough that third
+parties (e.g., OpenShift) will not need their own implementation, rather they
+can change the certs, token, etc., to be appropriate for their own servers,
+etc..
+
+Action items:
+* Split out of current client package into a new package. (@krousey)
+
+### RESTClient/request.go
+
+RESTClient consumes a Transport and a Codec (and optionally a group/version),
+and produces something that implements the interface currently in request.go.
+That is, with a RESTClient, you can write chains of calls like:
+
+`c.Get().Path(p).Param("name", "value").Do()`
+
+RESTClient is generically usable by any client for servers exposing REST-like
+semantics. It provides helpers that benefit those following api-conventions.md,
+but does not mandate them. It provides a higher level http interface that
+abstracts transport, wire serialization, retry logic, and error handling.
+Kubernetes-like constructs that deviate from standard HTTP should be bypassable.
+Every non-trivial call made to a remote restful API from Kubernetes code should
+go through a rest client.
+
+The group and version may be empty when constructing a RESTClient. This is valid
+for executing discovery commands. The group and version may be overridable with
+a chained function call.
+
+Ideally, no semantic behavior is built into RESTClient, and RESTClient will use
+the Codec it was constructed with for all semantic operations, including turning
+options objects into URL query parameters. Unfortunately, that is not true of
+today's RESTClient, which may have some semantic information built in. We will
+remove this.
+
+RESTClient should not make assumptions about the format of data produced or
+consumed by the Codec. Currently, it is JSON, but we want to support binary
+protocols in the future.
+
+The Codec would look something like this:
+
+```go
+type Codec interface {
+  Encode(runtime.Object) ([]byte, error)
+  Decode([]byte]) (runtime.Object, error)
+
+  // Used to version-control query parameters
+  EncodeParameters(optionsObject runtime.Object) (url.Values, error)
+
+  // Not included here since the client doesn't need it, but a corresponding
+  // DecodeParametersInto method would be available on the server.
+}
+```
+
+There should be one codec per version. RESTClient is *not* responsible for
+converting between versions; if a client wishes, they can supply a Codec that
+does that. But RESTClient will make the assumption that it's talking to a single
+group/version, and will not contain any conversion logic. (This is a slight
+change from the current state.)
+
+As with Transport, it is expected that 3rd party providers following the api
+conventions should be able to use RESTClient, and will not need to implement
+their own.
+
+Action items:
+* Split out of the current client package. (@krousey)
+* Possibly, convert to an interface (currently, it's a struct). This will allow
+  extending the error-checking monad that's currently in request.go up an
+  additional layer.
+* Switch from ParamX("x") functions to using types representing the collection
+  of parameters and the Codec for query parameter serialization.
+* Any other Kubernetes group specific behavior should also be removed from
+  RESTClient.
+
+### Mux layer
+
+(See TODO at end; this can probably be merged with the "client set" concept.)
+
+The client muxer layer has a map of group/version to cached RESTClient, and
+knows how to construct a new RESTClient in case of a cache miss (using the
+discovery client mentioned below). The ClientMux may need to deal with multiple
+transports pointing at differing destinations (e.g. OpenShift or other 3rd party
+provider API may be at a different location).
+
+When constructing a RESTClient generically, the muxer will just use the Codec
+the high-level dynamic client would use. Alternatively, the user should be able
+to pass in a Codec-- for the case where the correct types are compiled in.
+
+Tentative name: ClientMux
+
+Action items:
+* Move client cache out of kubectl libraries into a more general home.
+* TODO: a mux layer may not be necessary, depending on what needs to be cached.
+  If transports are cached already, and RESTClients are extremely light-weight,
+  there may not need to be much code at all in this layer.
+
+### High-level: Individual typed
+
+Our current high-level client allows you to write things like
+`c.Pods("namespace").Create(p)`; we will insert a level for the group.
+
+That is, the system will be:
+
+`clientset.GroupName().NamespaceSpecifier().Action()`
+
+Where:
+* `clientset` is a thing that holds multiple individually typed clients (see
+  below).
+* `GroupName()` returns the generated client that this section is about.
+* `NamespaceSpecifier()` may take a namespace parameter or nothing.
+* `Action` is one of Create/Get/Update/Delete/Watch, or appropriate actions
+  from the type's subresources.
+* It is TBD how we'll represent subresources and their actions. This is
+  inconsistent in the current clients, so we'll need to define a consistent
+  format. Possible choices:
+ * Insert a `.Subresource()` before the `.Action()`
+ * Flatten subresources, such that they become special Actions on the parent
+   resource.
+
+The types returned/consumed by such functions will be e.g. api/v1, NOT the
+current version inspecific types. The current internal-versioned client is
+inconvenient for users, as it does not protect them from having to recompile
+their code with every minor update. (We may continue to generate an
+internal-versioned client for our own use for a while, but even for our own
+components it probably makes sense to switch to specifically versioned clients.)
+
+We will provide this structure for each version of each group. It is infeasible
+to do this manually, so we will generate this. The generator will accept both
+swagger and the ordinary go types. The generator should operate on out-of-tree
+sources AND out-of-tree destinations, so it will be useful for consuming
+out-of-tree APIs and for others to build custom clients into their own
+repositories.
+
+Typed clients will be constructabale given a ClientMux; the typed constructor will use
+the ClientMux to find or construct an appropriate RESTClient. Alternatively, a
+typed client should be constructable individually given a config, from which it
+will be able to construct the appropriate RESTClient.
+
+Typed clients do not require any version negotiation. The server either supports
+the client's group/version, or it does not. However, there are ways around this:
+* If you want to use a typed client against a server's API endpoint and the
+  server's API version doesn't match the client's API version, you can construct
+  the client with a RESTClient using a Codec that does the conversion (this is
+  basically what our client does now).
+* Alternatively, you could use the dynamic client.
+
+Action items:
+* Move current typed clients into new directory structure (described below)
+* Finish client generation logic. (@caesarxuchao, @lavalamp)
+
+#### High-level, typed: Discovery
+
+A `DiscoveryClient` is necessary to discover the api groups, versions, and
+resources a server supports. It's constructable given a RESTClient. It is
+consumed by both the ClientMux and users who want to iterate over groups,
+versions, or resources. (Example: namespace controller.)
+
+The DiscoveryClient is *not* required if you already know the group/version of
+the resource you want to use: you can simply try the operation without checking
+first, which is lower-latency anyway as it avoids an extra round-trip.
+
+Action items:
+* Refactor existing functions to present a sane interface, as close to that
+  offered by the other typed clients as possible. (@caeserxuchao)
+* Use a RESTClient to make the necessary API calls.
+* Make sure that no discovery happens unless it is explicitly requested. (Make
+  sure SetKubeDefaults doesn't call it, for example.)
+
+### High-level: Dynamic
+
+The dynamic client lets users consume apis which are not compiled into their
+binary. It will provide the same interface as the typed client, but will take
+and return `runtime.Object`s instead of typed objects. There is only one dynamic
+client, so it's not necessary to generate it, although optionally we may do so
+depending on whether the typed client generator makes it easy.
+
+A dynamic client is constructable given a config, group, and version. It will
+use this to construct a RESTClient with a Codec which encodes/decodes to
+'Unstructured' `runtime.Object`s. The group and version may be from a previous
+invocation of a DiscoveryClient, or they may be known by other means.
+
+For now, the dynamic client will assume that a JSON encoding is allowed. In the
+future, if we have binary-only APIs (unlikely?), we can add that to the
+discovery information and construct an appropriate dynamic Codec.
+
+Action items:
+* A rudimentary version of this exists in kubectl's builder. It needs to be
+  moved to a more general place.
+* Produce a useful 'Unstructured' runtime.Object, which allows for easy
+  Object/ListMeta introspection.
+
+### High-level: Client Sets
+
+Because there will be multiple groups with multiple versions, we will provide an
+aggregation layer that combines multiple typed clients in a single object.
+
+We do this to:
+* Deliver a concrete thing for users to consume, construct, and pass around. We
+  don't want people making 10 typed clients and making a random system to keep
+  track of them.
+* Constrain the testing matrix. Users can generate a client set at their whim
+  against their cluster, but we need to make guarantees that the clients we
+  shipped with v1.X.0 will work with v1.X+1.0, and vice versa. That's not
+  practical unless we "bless" a particular version of each API group and ship an
+  official client set with earch release. (If the server supports 15 groups with
+  2 versions each, that's 2^15 different possible client sets. We don't want to
+  test all of them.)
+
+A client set is generated into its own package. The generator will take the list
+of group/versions to be included. Only one version from each group will be in
+the client set.
+
+A client set is constructable at runtime from either a ClientMux or a transport
+config (for easy one-stop-shopping).
+
+An example:
+
+```go
+import (
+  api_v1 "k8s.io/kubernetes/pkg/client/typed/generated/v1"
+  ext_v1beta1 "k8s.io/kubernetes/pkg/client/typed/generated/extensions/v1beta1"
+  net_v1beta1 "k8s.io/kubernetes/pkg/client/typed/generated/net/v1beta1"
+  "k8s.io/kubernetes/pkg/client/typed/dynamic"
+)
+
+type Client interface {
+  API() api_v1.Client
+  Extensions() ext_v1beta1.Client
+  Net() net_v1beta1.Client
+  // ... other typed clients here.
+
+  // Included in every set
+  Discovery() discovery.Client
+  GroupVersion(group, version string) dynamic.Client
+}
+```
+
+Note that a particular version is chosen for each group. It is a general rule
+for our API structure that no client need care about more than one version of
+each group at a time.
+
+This is the primary deliverable that people would consume. It is also generated.
+
+Action items:
+* This needs to be built. It will replace the ClientInterface that everyone
+  passes around right now.
+
+## Package Structure
+
+```
+pkg/client/
+----------/transport/     # transport & associated config
+----------/restclient/
+----------/clientmux/
+----------/typed/
+----------------/discovery/
+----------------/generated/
+--------------------------/<group>/
+----------------------------------/<version>/
+--------------------------------------------/<resource>.go
+----------------/dynamic/
+----------/clientsets/
+---------------------/release-1.1/
+---------------------/release-1.2/
+---------------------/the-test-set-you-just-generated/
+```
+
+`/clientsets/` will retain their contents until they reach their expire date.
+e.g., when we release v1.N, we'll remove clientset v1.(N-3). Clients from old
+releases live on and continue to work (i.e., are tested) without any interface
+changes for multiple releases, to give users time to transition.
+
+## Client Guarantees (and testing)
+
+Once we release a clientset, we will not make interface changes to it. Users of
+that client will not have to change their code until they are deliberately
+upgrading their import. We probably will want to generate some sort of stub test
+with a clienset, to ensure that we don't change the interface.
+
+
+<!-- BEGIN MUNGE: GENERATED_ANALYTICS -->
+[![Analytics](https://kubernetes-site.appspot.com/UA-36037335-10/GitHub/docs/proposals/client-package-structure.md?pixel)]()
+<!-- END MUNGE: GENERATED_ANALYTICS -->