Update kubectl in addon-manager to use HPA in autoscaling/v1

[MrHohn]

Addon-manager is broken since HPA objects were removed from extensions api group.

Came across the logs from the latest addon-manager on Jenkins:

INFO: == Entering periodical apply loop at 2017-02-16T17:33:37+0000 ==
error: error pruning namespaced object extensions/v1beta1, Kind=HorizontalPodAutoscaler: the server could not find the requested resource
WRN: == Failed to execute /usr/local/bin/kubectl  apply --namespace=kube-system -f /etc/kubernetes/addons     --prune=true -l kubernetes.io/cluster-service=true --recursive >/dev/null at 2017-02-16T17:33:38+0000. 2 tries remaining. ==
error: error pruning namespaced object extensions/v1beta1, Kind=HorizontalPodAutoscaler: the server could not find the requested resource
WRN: == Failed to execute /usr/local/bin/kubectl  apply --namespace=kube-system -f /etc/kubernetes/addons     --prune=true -l kubernetes.io/cluster-service=true --recursive >/dev/null at 2017-02-16T17:33:46+0000. 1 tries remaining. ==
error: error pruning namespaced object extensions/v1beta1, Kind=HorizontalPodAutoscaler: the server could not find the requested resource
WRN: == Failed to execute /usr/local/bin/kubectl  apply --namespace=kube-system -f /etc/kubernetes/addons     --prune=true -l kubernetes.io/cluster-service=true --recursive >/dev/null at 2017-02-16T17:33:53+0000. 0 tries remaining. ==
WRN: == Kubernetes addon update completed with errors at 2017-02-16T17:33:58+0000 ==

And notice this commit (f66679a) came in two weeks ago, which removed HorizontalPodAutoscaler from extensions/v1beta1.

Addon-manager is now partially functioning that it could successfully create and update addons, but will fail to prune objects, which means upgrade tests may mostly fail.

Pushed another version of addon-manager with kubectl v1.6.0-alpha.2 (release 2 days ago) for fixing, including below images:

• gcr.io/google-containers/kube-addon-manager:v6.4-alpha.2
• gcr.io/google-containers/kube-addon-manager-amd64:v6.4-alpha.2
• gcr.io/google-containers/kube-addon-manager-arm:v6.4-alpha.2
• gcr.io/google-containers/kube-addon-manager-arm64:v6.4-alpha.2
• gcr.io/google-containers/kube-addon-manager-ppc64le:v6.4-alpha.2
• gcr.io/google-containers/kube-addon-manager-s390x:v6.4-alpha.2

@mikedanese

cc @wojtek-t @shyamjvs

[k8s-reviewable]

This change is 

[jbeda]

Don't know this super well, but looks reasonable.

/lgtm

[gyliu513]

The path for kube-addon-manager.yaml was actually updated to

cluster/saltbase/salt/kube-addons/kube-addon-manager.yaml
test/kubemark/resources/manifests/kube-addon-manager.yaml

My personal thinking is that we can probably remove all of the comments here

    # When updating version also bump it in:
    # - cluster/images/hyperkube/static-pods/addon-manager-singlenode.json
    # - cluster/images/hyperkube/static-pods/addon-manager-multinode.json
    # - cluster/gce/coreos/kube-manifests/kube-addon-manager.yaml

As the comments is difficult to maintain and also I think that most developer can simply grep to get and replace all needed places.

[MrHohn]

@gyliu513 Removed the pointers in test/kubemark/resources/manifests/kube-addon-manager.yaml and updated the references in cluster/saltbase/salt/kube-addons/kube-addon-manager.yaml, which is expected to be the entry point when bumping up addon-manager.

Yeah these comments are messy, but it would be better to have it. At least people will know they may need to grep. I hope we will later have pre-submit checks to ensure files are in-synced (by putting specific lint comments in codes).

@jbeda Sorry for the change after lgtm...

[gyliu513]

fair enough, thanks.

[MrHohn]

@k8s-bot unit test this

[jbeda]

/lgtm

[gmarek]

/approve

[roberthbailey]

/approve

[k8s-github-robot]

[APPROVALNOTIFIER] This PR is APPROVED

The following people have approved this PR: MrHohn, gmarek, roberthbailey

Needs approval from an approver in each of these OWNERS Files:

• cluster/OWNERS [roberthbailey]
• test/kubemark/OWNERS [gmarek]

You can indicate your approval by writing /approve in a comment
You can cancel your approval by writing /approve cancel in a comment

[k8s-github-robot]

Automatic merge from submit-queue (batch tested with PRs 41349, 41532, 41256, 41587, 41657)