[Kubernetes] Expand condition support to remaining inputs
#7395
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
💚 Build Succeeded
Expand to view the summary
Build stats
Test stats 🧪
🤖 GitHub commentsExpand to view the GitHub comments
To re-run your PR in the CI, just comment with:
|
MichaelKatsoulis
requested review from
MichaelKatsoulis
and removed request for
gsantoro
gizas
reviewed
Aug 22, 2023
|
@BenB196 thank you for this. I also think we needed this. I added a small update for kubelet to show the condition to users Second and most important I believe that kube-state-metrics wont need a global condition, the opposite only specific ones per datastream. Main reason is that I would like to avoid users setting leader off in the global condition and keep leader flag open to all sub datastreams. So I would sugest to remove global condition from kube-state-metrics and introduce one per ksm datastream. Apart from that, the tests and all checks seem ok! Thank you again |
gizas
reviewed
Aug 22, 2023
Co-authored-by: Andrew Gizas <andreas.gkizas@elastic.co>
BenB196
commented
Aug 22, 2023
There was a problem hiding this comment.
I retested the Kubelet section as this was the only part the experienced change.
No top-level condition, not per stream condition
inputs:
- id: kubernetes/metrics-kubelet-dfd54600-4141-11ee-b801-0987c2093460
name: kubernetes-1
revision: 1
type: kubernetes/metrics
use_output: default
meta:
package:
name: kubernetes
version: 1.44.0
data_stream:
namespace: default
package_policy_id: dfd54600-4141-11ee-b801-0987c2093460
streams:
- id: >-
kubernetes/metrics-kubernetes.container-dfd54600-4141-11ee-b801-0987c2093460
data_stream:
dataset: kubernetes.container
type: metrics
metricsets:
- container
add_metadata: true
hosts:
- 'https://${env.NODE_NAME}:10250'
period: 10s
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
ssl.verification_mode: none
- id: >-
kubernetes/metrics-kubernetes.node-dfd54600-4141-11ee-b801-0987c2093460
data_stream:
dataset: kubernetes.node
type: metrics
metricsets:
- node
add_metadata: true
hosts:
- 'https://${env.NODE_NAME}:10250'
period: 10s
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
ssl.verification_mode: none
- id: kubernetes/metrics-kubernetes.pod-dfd54600-4141-11ee-b801-0987c2093460
data_stream:
dataset: kubernetes.pod
type: metrics
metricsets:
- pod
add_metadata: true
hosts:
- 'https://${env.NODE_NAME}:10250'
period: 10s
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
ssl.verification_mode: none
- id: >-
kubernetes/metrics-kubernetes.system-dfd54600-4141-11ee-b801-0987c2093460
data_stream:
dataset: kubernetes.system
type: metrics
metricsets:
- system
add_metadata: true
hosts:
- 'https://${env.NODE_NAME}:10250'
period: 10s
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
ssl.verification_mode: none
- id: >-
kubernetes/metrics-kubernetes.volume-dfd54600-4141-11ee-b801-0987c2093460
data_stream:
dataset: kubernetes.volume
type: metrics
metricsets:
- volume
add_metadata: true
hosts:
- 'https://${env.NODE_NAME}:10250'
period: 10s
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
ssl.verification_mode: noneTop level condition, and per stream condition
inputs:
- id: kubernetes/metrics-kubelet-dfd54600-4141-11ee-b801-0987c2093460
name: kubernetes-1
revision: 3
type: kubernetes/metrics
use_output: default
meta:
package:
name: kubernetes
version: 1.44.0
data_stream:
namespace: default
package_policy_id: dfd54600-4141-11ee-b801-0987c2093460
streams:
- id: >-
kubernetes/metrics-kubernetes.container-dfd54600-4141-11ee-b801-0987c2093460
data_stream:
dataset: kubernetes.container
type: metrics
period: 10s
condition: '${host.name} == ''testing'' and ${pod.name} == ''hello-world'''
add_metadata: true
hosts:
- 'https://${env.NODE_NAME}:10250'
ssl.verification_mode: none
metricsets:
- container
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
- id: >-
kubernetes/metrics-kubernetes.node-dfd54600-4141-11ee-b801-0987c2093460
data_stream:
dataset: kubernetes.node
type: metrics
period: 10s
condition: '${host.name} == ''testing'' and ${pod.name} == ''hello-world'''
add_metadata: true
hosts:
- 'https://${env.NODE_NAME}:10250'
ssl.verification_mode: none
metricsets:
- node
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
- id: kubernetes/metrics-kubernetes.pod-dfd54600-4141-11ee-b801-0987c2093460
data_stream:
dataset: kubernetes.pod
type: metrics
period: 10s
condition: '${host.name} == ''testing'' and ${pod.name} == ''hello-world'''
add_metadata: true
hosts:
- 'https://${env.NODE_NAME}:10250'
ssl.verification_mode: none
metricsets:
- pod
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
- id: >-
kubernetes/metrics-kubernetes.system-dfd54600-4141-11ee-b801-0987c2093460
data_stream:
dataset: kubernetes.system
type: metrics
period: 10s
condition: '${host.name} == ''testing'' and ${pod.name} == ''hello-world'''
add_metadata: true
hosts:
- 'https://${env.NODE_NAME}:10250'
ssl.verification_mode: none
metricsets:
- system
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
- id: >-
kubernetes/metrics-kubernetes.volume-dfd54600-4141-11ee-b801-0987c2093460
data_stream:
dataset: kubernetes.volume
type: metrics
period: 10s
condition: '${host.name} == ''testing'' and ${pod.name} == ''hello-world'''
add_metadata: true
hosts:
- 'https://${env.NODE_NAME}:10250'
ssl.verification_mode: none
metricsets:
- volume
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/tokenNo top level condition, per stream condition
inputs:
- id: kubernetes/metrics-kubelet-dfd54600-4141-11ee-b801-0987c2093460
name: kubernetes-1
revision: 4
type: kubernetes/metrics
use_output: default
meta:
package:
name: kubernetes
version: 1.44.0
data_stream:
namespace: default
package_policy_id: dfd54600-4141-11ee-b801-0987c2093460
streams:
- id: >-
kubernetes/metrics-kubernetes.container-dfd54600-4141-11ee-b801-0987c2093460
data_stream:
dataset: kubernetes.container
type: metrics
period: 10s
condition: '${pod.name} == ''hello-world'''
add_metadata: true
hosts:
- 'https://${env.NODE_NAME}:10250'
ssl.verification_mode: none
metricsets:
- container
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
- id: >-
kubernetes/metrics-kubernetes.node-dfd54600-4141-11ee-b801-0987c2093460
data_stream:
dataset: kubernetes.node
type: metrics
period: 10s
condition: '${pod.name} == ''hello-world'''
add_metadata: true
hosts:
- 'https://${env.NODE_NAME}:10250'
ssl.verification_mode: none
metricsets:
- node
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
- id: kubernetes/metrics-kubernetes.pod-dfd54600-4141-11ee-b801-0987c2093460
data_stream:
dataset: kubernetes.pod
type: metrics
period: 10s
condition: '${pod.name} == ''hello-world'''
add_metadata: true
hosts:
- 'https://${env.NODE_NAME}:10250'
ssl.verification_mode: none
metricsets:
- pod
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
- id: >-
kubernetes/metrics-kubernetes.system-dfd54600-4141-11ee-b801-0987c2093460
data_stream:
dataset: kubernetes.system
type: metrics
period: 10s
condition: '${pod.name} == ''hello-world'''
add_metadata: true
hosts:
- 'https://${env.NODE_NAME}:10250'
ssl.verification_mode: none
metricsets:
- system
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
- id: >-
kubernetes/metrics-kubernetes.volume-dfd54600-4141-11ee-b801-0987c2093460
data_stream:
dataset: kubernetes.volume
type: metrics
period: 10s
condition: '${pod.name} == ''hello-world'''
add_metadata: true
hosts:
- 'https://${env.NODE_NAME}:10250'
ssl.verification_mode: none
metricsets:
- volume
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/tokenTop level condition, no per stream condition
inputs:
- id: kubernetes/metrics-kubelet-dfd54600-4141-11ee-b801-0987c2093460
name: kubernetes-1
revision: 5
type: kubernetes/metrics
use_output: default
meta:
package:
name: kubernetes
version: 1.44.0
data_stream:
namespace: default
package_policy_id: dfd54600-4141-11ee-b801-0987c2093460
streams:
- id: >-
kubernetes/metrics-kubernetes.container-dfd54600-4141-11ee-b801-0987c2093460
data_stream:
dataset: kubernetes.container
type: metrics
period: 10s
condition: '${host.name} == ''testing'''
add_metadata: true
hosts:
- 'https://${env.NODE_NAME}:10250'
ssl.verification_mode: none
metricsets:
- container
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
- id: >-
kubernetes/metrics-kubernetes.node-dfd54600-4141-11ee-b801-0987c2093460
data_stream:
dataset: kubernetes.node
type: metrics
period: 10s
condition: '${host.name} == ''testing'''
add_metadata: true
hosts:
- 'https://${env.NODE_NAME}:10250'
ssl.verification_mode: none
metricsets:
- node
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
- id: kubernetes/metrics-kubernetes.pod-dfd54600-4141-11ee-b801-0987c2093460
data_stream:
dataset: kubernetes.pod
type: metrics
period: 10s
condition: '${host.name} == ''testing'''
add_metadata: true
hosts:
- 'https://${env.NODE_NAME}:10250'
ssl.verification_mode: none
metricsets:
- pod
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
- id: >-
kubernetes/metrics-kubernetes.system-dfd54600-4141-11ee-b801-0987c2093460
data_stream:
dataset: kubernetes.system
type: metrics
period: 10s
condition: '${host.name} == ''testing'''
add_metadata: true
hosts:
- 'https://${env.NODE_NAME}:10250'
ssl.verification_mode: none
metricsets:
- system
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
- id: >-
kubernetes/metrics-kubernetes.volume-dfd54600-4141-11ee-b801-0987c2093460
data_stream:
dataset: kubernetes.volume
type: metrics
period: 10s
condition: '${host.name} == ''testing'''
add_metadata: true
hosts:
- 'https://${env.NODE_NAME}:10250'
ssl.verification_mode: none
metricsets:
- volume
bearer_token_file: /var/run/secrets/kubernetes.io/serviceaccount/token
Comment on lines
+41
to
+43
| description: | | ||
| Condition to filter when to collect this input. See [Dynamic Input Configuration](https://www.elastic.co/guide/en/fleet/current/dynamic-input-configuration.html) for details. | ||
| Note: This condition value will be appended to the individual metric condition values by an "and" clause if defined. |
There was a problem hiding this comment.
I've expanded the description of this field a bit to highlight how this condition interacts with the individual metric conditions so people don't need to go looking/guess.
|
Hi @gizas, don't mean to pester, but was curious if there is anything blocking this PR from being reviewed/merged? |
gizas
approved these changes
Sep 18, 2023
|
Hi @MichaelKatsoulis @tetianakravchenko I see you're marked as a requested review on this PR. I don't mean to bother, but would it be possible to get the PR reviewed? Also, it looks like the test never was run on this PR. It would be nice to have this PR merged as it would greatly reduce some operational overhead. |
🌐 Coverage report
|
pierrehilbert
added
the
Team:Cloudnative-Monitoring
|
@MichaelKatsoulis @tetianakravchenko could you please have a look? |
|
Package kubernetes - 1.54.0 containing this change is available at https://epr.elastic.co/search?package=kubernetes |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Enhancement - This change expands the ability to set
conditionto the remaining inputs within the Kubernetes integration.The main driving factor behind this change is to allow for creating more dynamic Elastic Agent policies which can be used to monitor multiple different Kubernetes setups (ex: different distro's - RKE2, EKS, AKS, etc...; different IP schemas - IPv4 vs IPv6) from a single Elastic Agent policy that contains conditionally defined Kubernetes integration inputs.
Checklist
changelog.ymlfile.Author's Checklist
conditionfields added are hidden/advanced, they're targeted at more edge-case needs I believe, so I don't think it makes sense to show themconditionfieldconditionon each metric input individuallyconditionfieldconditionon each metric input individuallyconditionfield here is to be able to support host values that need to target either IPv4 or IPv6 (host values in IPv6 need to be wrapped in[]when also specifying a port)conditionfield is to be able to support different Kubernetes distro's where their audit log location is differentHow to test this PR locally
conditionvalues as neededconditionvalues are added appropriatelyBelow are configs which demonstrate that the
conditionfunctionality has been tested at a configuration level. I've broken the evidence down by input (group) as this is a somewhat large change.Kubelet API
---No `condition` set
`condition` set
kube-state-metrics
---No `condition` set, but leader election set
`condition` set, and leader election set
`condition` set, no leader election set
No `condition` set, no leader election set
Kubernetes API Server
---No `condition`, but leader election set
`condition` set, and leader election set
`condition` set, but leader election not set
No `condition` set, no leader election set
Kubernetes Proxy
---No `condition` set
`condition` set
Kubernetes Scheduler
---No `condition` set
`condition` set
Kubernetes controller-manager
---No `condition`
`condition` set
Kubernetes API Server
---No `condition`, but with leader election
`condition` set, and leader election set
`condition` set, but leader election not set
No `condition` set, and leader election not set
Kubernetes audit logs
---No `condition` set
`condition` set
Related issues
Screenshots