Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Kubernetes metadata #342

Merged
merged 1 commit into from Dec 4, 2018
Merged

Add Kubernetes metadata #342

merged 1 commit into from Dec 4, 2018

Conversation

axw
Copy link
Member

@axw axw commented Nov 30, 2018

Add Kubernetes metadata. We sniff pod metadata from /proc/self/cgroup, and then add or override any values specified through the environment variables KUBERNETES_NAMESPACE, KUBERNETES_NODE_NAME, KUBERNETES_POD_NAME, and KUBERNETES_POD_UID. These can be populated using the Kubernetes Downward API.

Through /proc/self/cgroup we are able to sniff the container ID and pod UID. We also assume that the hostname is the pod name, which is the default. If the hostname has been explicitly set, then the user can use the Downward API as mentioned above to set the correct pod name.

Closes #329

@codecov-io
Copy link

codecov-io commented Nov 30, 2018

Codecov Report

Merging #342 into master will increase coverage by 2.4%.
The diff coverage is 40.38%.

Impacted file tree graph

@@            Coverage Diff            @@
##           master     #342     +/-   ##
=========================================
+ Coverage   81.14%   83.54%   +2.4%     
=========================================
  Files          98       98             
  Lines        5891     6009    +118     
=========================================
+ Hits         4780     5020    +240     
- Misses        840      923     +83     
+ Partials      271       66    -205
Impacted Files Coverage Δ
model/marshal_fastjson.go 75.12% <0%> (+2.15%) ⬆️
internal/apmhostutil/container.go 100% <100%> (ø) ⬆️
utils.go 80.68% <44.82%> (-16.1%) ⬇️
internal/apmhostutil/container_linux.go 85.18% <85.18%> (ø)
modelwriter.go 94.16% <0%> (+0.83%) ⬆️
internal/radix/radix.go 39.92% <0%> (+0.92%) ⬆️
internal/sqlscanner/scanner.go 96.82% <0%> (+1.58%) ⬆️
metrics.go 91.66% <0%> (+1.66%) ⬆️
... and 34 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0ac0db8...3c7e66d. Read the comment docs.

@axw
Copy link
Member Author

axw commented Nov 30, 2018

While testing, I realised that the container.id wasn't being populated. That's because the cgroup paths are different under k8s. They look like this:

1:name=systemd:/kubepods/besteffort/pode9b90526-f47d-11e8-b2a5-080027b9f4fb/d87d73916efef57277e02f38eea58495a5e3687a494e34ffa8e14fc2b8a52dc3

The last segment is the Docker container ID. Obviously only holds if Docker is the container runtime being used. The second-to-last segment is the pod UID, prefixed with "pod". I also realised we can get the pod name just from the hostname.

Need to run some more tests, with the following variables:

  • kubelet cgroup driver: cgroupfs vs. systemd
  • alternative container runtime?

@axw axw changed the title Add Kubernetes metadata from env vars Add Kubernetes metadata Dec 3, 2018
@axw axw force-pushed the k8s-metadata branch 2 times, most recently from 4a23929 to 9615697 Compare December 3, 2018 07:00
@axw axw force-pushed the k8s-metadata branch 2 times, most recently from cb27f5b to 3cd3fa4 Compare December 4, 2018 08:30
We sniff kubernetes pod metadata from /proc/self/cgroup,
and add or override any values specified through the environment
variables KUBERNETES_{NAMESPACE,NODE_NAME,POD_NAME,POD_UID}.

Also, stop checking for "docker" in the cgroup path. The
64-bit hex ID check should be sufficient to avoid false
positives. We only do this check for non-k8s environments.
@axw axw merged commit 949be0f into elastic:master Dec 4, 2018
@axw axw deleted the k8s-metadata branch December 4, 2018 10:08
beniwohli added a commit to beniwohli/apm-agent-python that referenced this pull request Dec 19, 2018
beniwohli added a commit to beniwohli/apm-agent-python that referenced this pull request Jan 2, 2019
beniwohli added a commit to elastic/apm-agent-python that referenced this pull request Jan 3, 2019
parsing uses regexes heavily inspired by elastic/apm-agent-go#342

(as in, pretty much copied from elastic/apm-agent-go#342)

closes #352
fixes #346
fixes #347
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants