Customize Kops Platform

[gambol99]

Kubelet API (kubernetes#3125)

A while back options to permit secure kube-apiserver to kubelet api was PR2381 using the server.cert and server.key as testing grounds. This PR formalizes the options and generates a client certificate on their behalf (note, the server{.cert,key} can no longer be used post 1.7 as the certificate usage is checked i.e. it's not using a client cert). The users now only need to add anonymousAuth: false to enable secure api to kubelet. I'd like to make this default to all new builds i'm not sure where to place it.

• updated the security.md to reflect the changes
• issue a new client kubelet-api certificate used to secure authorize comms between api and kubelet
• fixed any formatting issues i came across on the journey

Etcd TLS (kubernetes#3114)

The current implementation does not put any transport security on the etcd cluster. The PR provides and optional flag to enable TLS the etcd cluster

• cleaned up and fixed any formatting issues on the journey
• added two new certificates (server/client) for etcd peers and a client certificate for kubeapi and others perhaps (perhaps calico?)
• disabled the protokube service for nodes completely is not required; note this was first raised in Disable Protokube Options - Nodes kubernetes/kops#3091, but figured it would be easier to place in here given the relation
• updated protokube codebase to reflect the changes, removing the master option as its no longer required
• added additional integretion tests for the protokube manifests;
• note, still need to add documentation, but opening the PR to get feedback
• one outstanding issue is the migration from http -> https for preexisting clusters, i'm gonna hit the coreos board to ask for the best options

Configuration Detection (kubernetes#3120)

Related to kubernetes#3076

Some cluster changes such as component config modifications are not picked up when performing updates (nodes are not marked as NEEDUPDATE). This change introduces the ability to:
- Include certain cluster specs within the node user data file (enableClusterSpecInUserData: true)
- Encode the cluster spec string before placing within the user data file (enableClusterSpecInUserData: true)
- The above flags default to false so shouldn't cause any changes to existing clusters.

Cluster Hook Enhancement (kubernetes#3063)

The current implementation is presently limited to docker exec, without ordering or any bells and whistles. This PR extends the functionality of the hook spec by;

• adds ordering to the hooks, with users able to set the requires and before of the unit
• cleaned up the manifest code, added tests and permit setting a section raw
• added the ability to filter hooks via master and node roles
• updated the documentation to reflect the changes
• extending the hooks to permit adding hooks per instancegroup as well cluster
• @note, instanceGroup are permitted to override the cluster wide one for ease of testing
• on the journey tried to fix an go idioms such as import ordering, comments for global export etc
• @question: v1alpha1 doesn't appear to have Subnet fields, are these different version being used anywhere?

Cluster Inline File assets (kubernetes#3090)

The current implementation does not make it ease to fully customize nodes before kube install. This PR adds the ability to include file assets in the cluster and instaneGroup spec which can be consumed by nodeup. Allowing those whom need (i.e. me :-)) greater flexibilty around their nodes. @note, nothing is enforced, so unless you've specified anything everything is as the same

• updated the cluster_spec.md to reflect the changes
• permit users to place inline files into the cluster and instance group specs
• added the ability to template the files, the Cluster and InstanceGroup specs are passed into context
• cleaned up and missed comment, unordered imports etc along the journey

notes: In addition to this; need to look at the detecting the changes in the cluster and instance group spec. Think out loud perhaps using a last_known_configuration annotation, similar to kubernetes

Replace and Create Command (kubernetes#3090)

The current 'kops replace' fails if the resource does not exist, which is annoying if you want to use the feature to drive your CI. This PR adds a --create option to create any resource which does not exist. At the moment we limit this to instanceGroups only. I'd also like to see this command perhaps be renamed to kops apply?

• added a --create command line option to the replace command to create unprovisioned resources

Fixes

• fixed some conflicts within the merge
• fixed up the go formatting error which were in the merges