New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update the stackdriver agents yaml to include a deployment for cluster level resources #62043

Merged
merged 1 commit into from Apr 5, 2018

Conversation

@supriyagarg
Contributor

supriyagarg commented Apr 2, 2018

What this PR does / why we need it: This PR introduces a Deployment based on the Stackdriver Metadata Agent, that will allow users to collect cluster level metadata like unscheduled pods, and services. All the services in a cluster are written against the "k8s_cluster" resource.

Extend the Stackdriver Metadata Agent by adding a new Deployment for ingesting unscheduled pods, and services.  
@k8s-ci-robot

This comment has been minimized.

Contributor

k8s-ci-robot commented Apr 2, 2018

Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please follow instructions at https://git.k8s.io/community/CLA.md#the-contributor-license-agreement to sign the CLA.

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify. Thanks.


Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@zouyee

This comment has been minimized.

Member

zouyee commented Apr 3, 2018

/ok-to-test

@@ -30,6 +30,8 @@ spec:
- image: gcr.io/stackdriver-agents/stackdriver-metadata-agent:{{ metadata_agent_version }}
imagePullPolicy: IfNotPresent
name: metadata-agent
command: ["/opt/stackdriver/metadata/sbin/metadatad"]

This comment has been minimized.

@kawych

kawych Apr 3, 2018

Contributor

Please use list syntax consistent with the rest of the file (each element in separate line preceded with a dash).

This comment has been minimized.

@supriyagarg

supriyagarg Apr 3, 2018

Contributor

Done. Merged the args into the command list.

@@ -2156,11 +2156,15 @@ EOF
[[ "${METADATA_AGENT_VERSION:-}" != "" ]]; then
metadata_agent_cpu_request="${METADATA_AGENT_CPU_REQUEST:-40m}"
metadata_agent_memory_request="${METADATA_AGENT_MEMORY_REQUEST:-50Mi}"
metadata_agent_cluster_level_cpu_request="${METADATA_AGENT_CLUSTER_LEVEL_CPU_REQUEST:-40m}"

This comment has been minimized.

@kawych

kawych Apr 3, 2018

Contributor

@supriyagarg @x13n
Do we have any more up-to-date numbers, so that we can set reasonable defaults?

This comment has been minimized.

@supriyagarg

supriyagarg Apr 3, 2018

Contributor

I don't have any up-to-date numbers. Let me check with others in the agents team.

This comment has been minimized.

@kawych

kawych Apr 4, 2018

Contributor

Let's not block this PR, but set reasonable defaults separately

This comment has been minimized.

@x13n

x13n Apr 4, 2018

Member

This is way too high to be acceptable (but the same goes for the DaemonSet one).

@supriyagarg

Responded to comments, and udpated implementation to use a ConfigMap

@@ -30,6 +30,8 @@ spec:
- image: gcr.io/stackdriver-agents/stackdriver-metadata-agent:{{ metadata_agent_version }}
imagePullPolicy: IfNotPresent
name: metadata-agent
command: ["/opt/stackdriver/metadata/sbin/metadatad"]

This comment has been minimized.

@supriyagarg

supriyagarg Apr 3, 2018

Contributor

Done. Merged the args into the command list.

@@ -2156,11 +2156,15 @@ EOF
[[ "${METADATA_AGENT_VERSION:-}" != "" ]]; then
metadata_agent_cpu_request="${METADATA_AGENT_CPU_REQUEST:-40m}"
metadata_agent_memory_request="${METADATA_AGENT_MEMORY_REQUEST:-50Mi}"
metadata_agent_cluster_level_cpu_request="${METADATA_AGENT_CLUSTER_LEVEL_CPU_REQUEST:-40m}"

This comment has been minimized.

@supriyagarg

supriyagarg Apr 3, 2018

Contributor

I don't have any up-to-date numbers. Let me check with others in the agents team.

@kawych

This comment has been minimized.

Contributor

kawych commented Apr 4, 2018

LGTM, please squash the commits.

@supriyagarg

This comment has been minimized.

Contributor

supriyagarg commented Apr 4, 2018

Thanks - bumped the version to 0.18.2 and squashed the commits.

@supriyagarg

This comment has been minimized.

Contributor

supriyagarg commented Apr 4, 2018

/retest

@k8s-ci-robot k8s-ci-robot added size/L and removed size/M labels Apr 4, 2018

@supriyagarg

This comment has been minimized.

Contributor

supriyagarg commented Apr 4, 2018

Update: bumped the version to 0.19.1, and added a liveness probes to both the daemonset and the deployment

@supriyagarg

This comment has been minimized.

Contributor

supriyagarg commented Apr 4, 2018

/retest

@kawych

This comment has been minimized.

Contributor

kawych commented Apr 5, 2018

/lgtm

@supriyagarg

This comment has been minimized.

Contributor

supriyagarg commented Apr 5, 2018

/assign @MaciekPytel

@x13n

This comment has been minimized.

Member

x13n commented Apr 5, 2018

One thing though: Please add the new env variables to PROVIDER_VARS. Without this it won't be possible to use them.

@MaciekPytel

This comment has been minimized.

Contributor

MaciekPytel commented Apr 5, 2018

@supriyagarg Shouldn't this have a release note (especially if you're planning to cherry-pick it)?

@MaciekPytel

This comment has been minimized.

Contributor

MaciekPytel commented Apr 5, 2018

/approve
/hold

Please remove hold after you address comments by @x13n.

@supriyagarg

This comment has been minimized.

Contributor

supriyagarg commented Apr 5, 2018

Just fixed a typo in the env vars: CLUSETER -> CLUSTER in one place.
The metadata agent env variables are not provider specific - do they still need to be added to PROVIDER_VARS to be used?

@x13n

This comment has been minimized.

Member

x13n commented Apr 5, 2018

@x13n

This comment has been minimized.

Member

x13n commented Apr 5, 2018

/lgtm

@k8s-ci-robot k8s-ci-robot removed the lgtm label Apr 5, 2018

@supriyagarg

This comment has been minimized.

Contributor

supriyagarg commented Apr 5, 2018

Squashed the commits.

@kawych

This comment has been minimized.

Contributor

kawych commented Apr 5, 2018

/lgtm
/hold cancel

@k8s-ci-robot k8s-ci-robot added lgtm and removed do-not-merge/hold labels Apr 5, 2018

@k8s-ci-robot

This comment has been minimized.

Contributor

k8s-ci-robot commented Apr 5, 2018

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kawych, MaciekPytel, supriyagarg, x13n

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@supriyagarg

This comment has been minimized.

Contributor

supriyagarg commented Apr 5, 2018

/retest

1 similar comment
@supriyagarg

This comment has been minimized.

Contributor

supriyagarg commented Apr 5, 2018

/retest

@k8s-ci-robot

This comment has been minimized.

Contributor

k8s-ci-robot commented Apr 5, 2018

@supriyagarg: The following tests failed, say /retest to rerun them all:

Test name Commit Details Rerun command
pull-kubernetes-e2e-gce-100-performance ed65b0d link /test pull-kubernetes-e2e-gce-100-performance
pull-kubernetes-e2e-gke e350c46 link /test pull-kubernetes-e2e-gke

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@k8s-merge-robot

This comment has been minimized.

Contributor

k8s-merge-robot commented Apr 5, 2018

Automatic merge from submit-queue (batch tested with PRs 62043, 62168). If you want to cherry-pick this change to another branch, please follow the instructions here.

@k8s-merge-robot k8s-merge-robot merged commit 405ea2b into kubernetes:master Apr 5, 2018

13 of 14 checks passed

pull-kubernetes-e2e-gke Job failed.
Details
Submit Queue Queued to run github e2e tests a second time.
Details
cla/linuxfoundation supriyagarg authorized
Details
pull-kubernetes-bazel-build Job succeeded.
Details
pull-kubernetes-bazel-test Job succeeded.
Details
pull-kubernetes-cross Skipped
pull-kubernetes-e2e-gce Job succeeded.
Details
pull-kubernetes-e2e-gce-device-plugin-gpu Job succeeded.
Details
pull-kubernetes-e2e-kops-aws Job succeeded.
Details
pull-kubernetes-integration Job succeeded.
Details
pull-kubernetes-kubemark-e2e-gce Job succeeded.
Details
pull-kubernetes-node-e2e Job succeeded.
Details
pull-kubernetes-typecheck Job succeeded.
Details
pull-kubernetes-verify Job succeeded.
Details

@supriyagarg supriyagarg deleted the supriyagarg:metadata-agent-cluster-deployment branch Apr 5, 2018

k8s-merge-robot added a commit that referenced this pull request Apr 6, 2018

Merge pull request #62174 from supriyagarg/automated-cherry-pick-of-#…
…62043-upstream-release-1.10

Automatic merge from submit-queue.

Automated cherry pick of #62043: Update the stackdriver agents yaml to include a deployment

Cherry pick of #62043 on release-1.10.

#62043: Update the stackdriver agents yaml to include a deployment
@@ -7,6 +7,19 @@ metadata:
kubernetes.io/cluster-service: "true"
addonmanager.kubernetes.io/mode: Reconcile
---
apiVersion: v1
kind: ConfigMap
metadata:

This comment has been minimized.

@kawych

kawych Apr 20, 2018

Contributor

This ConfigMap is missing labels for addon manager:
kubernetes.io/cluster-service: "true"
addonmanager.kubernetes.io/mode: Reconcile

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment