Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When running from within a k8s pod, use the k8s API to get the cgroup name #5576

Merged
merged 12 commits into from Mar 13, 2019
Merged

Conversation

cakrit
Copy link
Contributor

@cakrit cakrit commented Mar 7, 2019

Summary

Fixes #5387 (just the pod/container monitoring part, presentation will be a different issue)
Fixes #3520 (again, just wrt naming, presentation will be a new issue).

k8s don't necessarily have a docker executable available. Use the k8s pod API to get the pod name.
Used for cgroup naming.
Disable cgroups that are not returned by the k8s API (e.g. pause containers).

@cakrit cakrit added the area/collectors Everything related to data collection label Mar 7, 2019
@cakrit cakrit requested a review from paulfantom March 7, 2019 17:35
@@ -116,13 +133,16 @@ for CONFIG in "${NETDATA_USER_CONFIG_DIR}/cgroups-names.conf" "${NETDATA_STOCK_C
fi
done

if [ -z "${NAME}" ] && [ -n "${KUBERNETES_SERVICE_HOST}" ] && [ -n "${KUBERNETES_PORT_443_TCP_PORT}" ]; then
Copy link
Contributor

@paulfantom paulfantom Mar 7, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

KUBERNETES_PORT_443_TCP_PORT -> KUBERNETES_API_PORT

No need to copy-paste from k8s docs when it doesn't improve UX.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't seen KUBERNETES_API_PORT in any official docs, or even mentioned in issues. How can I trust it to be the https port?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It worked on all tests with this, so leaving it as is.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It worked on all tests with this, so leaving it as is.

@cakrit cakrit added this to In progress in Kubernetes monitoring via automation Mar 8, 2019
@cakrit
Copy link
Contributor Author

cakrit commented Mar 8, 2019

This won't be considered complete until #3520 is implemented (at least to the point of correctly showing only the relevant cgroups, with appropriate names).

@netdatabot netdatabot added the area/packaging Packaging and operating systems support label Mar 9, 2019
@cakrit cakrit changed the title [WIP] When running from within a k8s pod, use the k8s API to get the pod name When running from within a k8s pod, use the k8s API to get the cgroup name Mar 12, 2019
@cakrit cakrit requested a review from mfundul as a code owner March 13, 2019 10:47
Copy link
Member

@ktsaou ktsaou left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably do something about the jq command to make sure it is installed with netdata.

I mean, to be installed by kickstart.sh and netdata.spec.

@cakrit cakrit merged commit b7998ec into netdata:master Mar 13, 2019
Kubernetes monitoring automation moved this from In progress to Done Mar 13, 2019
@cakrit cakrit deleted the k8s-1 branch March 21, 2019 17:55
@woosley
Copy link
Contributor

woosley commented Jun 2, 2019

is this change just for netdata running in a k8s Pod? what about the situation when netdata is running directly on the VM with kubernetes running?

I have this situation so now the cgroup names are not mapped to the human readable docker container name.

@cakrit
Copy link
Contributor Author

cakrit commented Jun 3, 2019

jq is only uses in k8s. For the problem you have, please open a new issue with the details and we'll be happy to help. The log contains many entries from cgroup-name.sh, so please include info from the log as well.

jackyhuang85 pushed a commit to jackyhuang85/netdata that referenced this pull request Jan 1, 2020
… name (netdata#5576)

* When running from within a k8s pod, use the k8s API to get the pod name

* Check the last part of an underscore-delimited or slash-delimited container id

* Add a docker image builder that adds a single image to a user-specified registry, for use in k8s

* When running in k8s, disable cgroups that the pod API does not return

* Use longer name for k8s containers

* Add reference to build-test.sh to packaging/docker/README.md

* Anonymous statistics should not break when /proc/1/sched is not available
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/collectors Everything related to data collection area/packaging Packaging and operating systems support
Projects
No open projects
Development

Successfully merging this pull request may close these issues.

Kubernetes pod/containers monitoring Kubernetes Monitoring Improvement Ideas
6 participants