Update the stackdriver agents yaml to include a deployment for cluster level resources #62043

supriyagarg · 2018-04-02T22:40:13Z

What this PR does / why we need it: This PR introduces a Deployment based on the Stackdriver Metadata Agent, that will allow users to collect cluster level metadata like unscheduled pods, and services. All the services in a cluster are written against the "k8s_cluster" resource.

Extend the Stackdriver Metadata Agent by adding a new Deployment for ingesting unscheduled pods, and services.

k8s-ci-robot · 2018-04-02T22:40:16Z

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA.

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.

If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
If you signed the CLA as a corporation, please sign in with your organization's credentials at https://identity.linuxfoundation.org/projects/cncf to be authorized.
If you have done the above and are still having issues with the CLA being reported as unsigned, please email the CNCF helpdesk: helpdesk@rt.linuxfoundation.org

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

zouyee · 2018-04-03T03:06:44Z

/ok-to-test

kawych · 2018-04-03T14:55:15Z

cluster/addons/metadata-agent/stackdriver/metadata-agent.yaml

@@ -30,6 +30,8 @@ spec:
      - image: gcr.io/stackdriver-agents/stackdriver-metadata-agent:{{ metadata_agent_version }}
        imagePullPolicy: IfNotPresent
        name: metadata-agent
+        command: ["/opt/stackdriver/metadata/sbin/metadatad"]


Please use list syntax consistent with the rest of the file (each element in separate line preceded with a dash).

Done. Merged the args into the command list.

kawych · 2018-04-03T14:55:18Z

cluster/gce/gci/configure-helper.sh

@@ -2156,11 +2156,15 @@ EOF
       [[ "${METADATA_AGENT_VERSION:-}" != "" ]]; then
      metadata_agent_cpu_request="${METADATA_AGENT_CPU_REQUEST:-40m}"
      metadata_agent_memory_request="${METADATA_AGENT_MEMORY_REQUEST:-50Mi}"
+      metadata_agent_cluster_level_cpu_request="${METADATA_AGENT_CLUSTER_LEVEL_CPU_REQUEST:-40m}"


@supriyagarg @x13n
Do we have any more up-to-date numbers, so that we can set reasonable defaults?

I don't have any up-to-date numbers. Let me check with others in the agents team.

Let's not block this PR, but set reasonable defaults separately

This is way too high to be acceptable (but the same goes for the DaemonSet one).

supriyagarg

Responded to comments, and udpated implementation to use a ConfigMap

supriyagarg · 2018-04-03T17:05:26Z

cluster/addons/metadata-agent/stackdriver/metadata-agent.yaml

@@ -30,6 +30,8 @@ spec:
      - image: gcr.io/stackdriver-agents/stackdriver-metadata-agent:{{ metadata_agent_version }}
        imagePullPolicy: IfNotPresent
        name: metadata-agent
+        command: ["/opt/stackdriver/metadata/sbin/metadatad"]


Done. Merged the args into the command list.

supriyagarg · 2018-04-03T17:06:14Z

cluster/gce/gci/configure-helper.sh

@@ -2156,11 +2156,15 @@ EOF
       [[ "${METADATA_AGENT_VERSION:-}" != "" ]]; then
      metadata_agent_cpu_request="${METADATA_AGENT_CPU_REQUEST:-40m}"
      metadata_agent_memory_request="${METADATA_AGENT_MEMORY_REQUEST:-50Mi}"
+      metadata_agent_cluster_level_cpu_request="${METADATA_AGENT_CLUSTER_LEVEL_CPU_REQUEST:-40m}"


I don't have any up-to-date numbers. Let me check with others in the agents team.

kawych · 2018-04-04T08:42:10Z

LGTM, please squash the commits.

supriyagarg · 2018-04-04T20:54:31Z

Thanks - bumped the version to 0.18.2 and squashed the commits.

supriyagarg · 2018-04-04T21:29:04Z

/retest

supriyagarg · 2018-04-04T23:35:39Z

Update: bumped the version to 0.19.1, and added a liveness probes to both the daemonset and the deployment

supriyagarg · 2018-04-04T23:49:20Z

/retest

kawych · 2018-04-05T08:42:39Z

/lgtm

supriyagarg · 2018-04-05T10:47:52Z

/assign @MaciekPytel

MaciekPytel · 2018-04-05T12:41:02Z

@supriyagarg Shouldn't this have a release note (especially if you're planning to cherry-pick it)?

MaciekPytel · 2018-04-05T13:04:56Z

/approve
/hold

Please remove hold after you address comments by @x13n.

supriyagarg · 2018-04-05T13:30:56Z

Just fixed a typo in the env vars: CLUSETER -> CLUSTER in one place.
The metadata agent env variables are not provider specific - do they still need to be added to PROVIDER_VARS to be used?

x13n · 2018-04-05T13:52:57Z

They can be added directly to https://github.com/kubernetes/kubernetes/blob/master/cluster/gce/util.sh

x13n · 2018-04-05T13:58:40Z

/lgtm

…r level resources

supriyagarg · 2018-04-05T14:11:16Z

Squashed the commits.

kawych · 2018-04-05T14:12:04Z

/lgtm
/hold cancel

k8s-ci-robot · 2018-04-05T14:12:13Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kawych, MaciekPytel, supriyagarg, x13n

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~cluster/addons/metadata-agent/OWNERS~~ [kawych]
~~cluster/gce/OWNERS~~ [MaciekPytel]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

supriyagarg · 2018-04-05T15:22:02Z

/retest

supriyagarg · 2018-04-05T15:47:53Z

/retest

k8s-ci-robot · 2018-04-05T16:01:53Z

@supriyagarg: The following tests failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
pull-kubernetes-e2e-gce-100-performance	ed65b0df3fbea4f1a1a3f410ce7f6dd10bbac48a	link	`/test pull-kubernetes-e2e-gce-100-performance`
pull-kubernetes-e2e-gke	`e350c46`	link	`/test pull-kubernetes-e2e-gke`

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

k8s-github-robot · 2018-04-05T16:07:03Z

Automatic merge from submit-queue (batch tested with PRs 62043, 62168). If you want to cherry-pick this change to another branch, please follow the instructions here.

…62043-upstream-release-1.10 Automatic merge from submit-queue. Automated cherry pick of #62043: Update the stackdriver agents yaml to include a deployment Cherry pick of #62043 on release-1.10. #62043: Update the stackdriver agents yaml to include a deployment

kawych · 2018-04-20T14:15:34Z

cluster/addons/metadata-agent/stackdriver/metadata-agent.yaml

@@ -7,6 +7,19 @@ metadata:
    kubernetes.io/cluster-service: "true"
    addonmanager.kubernetes.io/mode: Reconcile
 ---
+apiVersion: v1
+kind: ConfigMap
+metadata:


This ConfigMap is missing labels for addon manager:
kubernetes.io/cluster-service: "true"
addonmanager.kubernetes.io/mode: Reconcile

k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Apr 2, 2018

k8s-ci-robot added cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Apr 2, 2018

k8s-ci-robot requested review from bowei and kawych April 2, 2018 22:40

k8s-ci-robot removed the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Apr 3, 2018

kawych reviewed Apr 3, 2018

View reviewed changes

supriyagarg commented Apr 3, 2018

View reviewed changes

supriyagarg force-pushed the metadata-agent-cluster-deployment branch from 69c71ac to 1ad414d Compare April 4, 2018 20:53

supriyagarg force-pushed the metadata-agent-cluster-deployment branch from 1ad414d to 526daf0 Compare April 4, 2018 23:31

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Apr 4, 2018

supriyagarg force-pushed the metadata-agent-cluster-deployment branch from 526daf0 to ed65b0d Compare April 4, 2018 23:33

k8s-ci-robot assigned kawych Apr 5, 2018

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 5, 2018

k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed release-note-none Denotes a PR that doesn't merit a release note. labels Apr 5, 2018

k8s-ci-robot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Apr 5, 2018

supriyagarg force-pushed the metadata-agent-cluster-deployment branch from ed65b0d to d44c1e9 Compare April 5, 2018 13:25

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 5, 2018

k8s-ci-robot assigned x13n Apr 5, 2018

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 5, 2018

Update the stackdriver agents yaml to include a deployment for cluste…

e350c46

…r level resources

supriyagarg force-pushed the metadata-agent-cluster-deployment branch from b0b8634 to e350c46 Compare April 5, 2018 14:10

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Apr 5, 2018

k8s-ci-robot added lgtm "Looks good to me", indicates that a PR is ready to be merged. and removed do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. labels Apr 5, 2018

k8s-github-robot merged commit 405ea2b into kubernetes:master Apr 5, 2018

supriyagarg deleted the metadata-agent-cluster-deployment branch April 5, 2018 16:40

supriyagarg mentioned this pull request Apr 5, 2018

Automated cherry pick of #62043: Update the stackdriver agents yaml to include a deployment #62174

Merged

kawych reviewed Apr 20, 2018

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update the stackdriver agents yaml to include a deployment for cluster level resources #62043

Update the stackdriver agents yaml to include a deployment for cluster level resources #62043

supriyagarg commented Apr 2, 2018 •

edited

k8s-ci-robot commented Apr 2, 2018

zouyee commented Apr 3, 2018

kawych Apr 3, 2018

supriyagarg Apr 3, 2018

kawych Apr 3, 2018

supriyagarg Apr 3, 2018

kawych Apr 4, 2018

x13n Apr 4, 2018

supriyagarg left a comment

supriyagarg Apr 3, 2018

supriyagarg Apr 3, 2018

kawych commented Apr 4, 2018

supriyagarg commented Apr 4, 2018

supriyagarg commented Apr 4, 2018

supriyagarg commented Apr 4, 2018 •

edited

supriyagarg commented Apr 4, 2018

kawych commented Apr 5, 2018

supriyagarg commented Apr 5, 2018

MaciekPytel commented Apr 5, 2018

MaciekPytel commented Apr 5, 2018

supriyagarg commented Apr 5, 2018

x13n commented Apr 5, 2018

x13n commented Apr 5, 2018

supriyagarg commented Apr 5, 2018

kawych commented Apr 5, 2018

k8s-ci-robot commented Apr 5, 2018

supriyagarg commented Apr 5, 2018

supriyagarg commented Apr 5, 2018

k8s-ci-robot commented Apr 5, 2018

k8s-github-robot commented Apr 5, 2018

kawych Apr 20, 2018

Update the stackdriver agents yaml to include a deployment for cluster level resources #62043

Update the stackdriver agents yaml to include a deployment for cluster level resources #62043

Conversation

supriyagarg commented Apr 2, 2018 • edited

k8s-ci-robot commented Apr 2, 2018

zouyee commented Apr 3, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

supriyagarg left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kawych commented Apr 4, 2018

supriyagarg commented Apr 4, 2018

supriyagarg commented Apr 4, 2018

supriyagarg commented Apr 4, 2018 • edited

supriyagarg commented Apr 4, 2018

kawych commented Apr 5, 2018

supriyagarg commented Apr 5, 2018

MaciekPytel commented Apr 5, 2018

MaciekPytel commented Apr 5, 2018

supriyagarg commented Apr 5, 2018

x13n commented Apr 5, 2018

x13n commented Apr 5, 2018

supriyagarg commented Apr 5, 2018

kawych commented Apr 5, 2018

k8s-ci-robot commented Apr 5, 2018

supriyagarg commented Apr 5, 2018

supriyagarg commented Apr 5, 2018

k8s-ci-robot commented Apr 5, 2018

k8s-github-robot commented Apr 5, 2018

Choose a reason for hiding this comment

supriyagarg commented Apr 2, 2018 •

edited

supriyagarg commented Apr 4, 2018 •

edited