Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add an example of autodiscover to deployment manifests #8029

Merged
merged 2 commits into from Aug 21, 2018
Merged

Add an example of autodiscover to deployment manifests #8029

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
00c9276
Add an example of autodiscover to deployment manifests
Aug 20, 2018
8b9182c
Use downward API to get node name
Aug 21, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 6 additions & 0 deletions deploy/kubernetes/filebeat-kubernetes.yaml
View file
Open in desktop
Expand Up @@ -19,6 +19,12 @@ data:
# Reload module configs as they change:
reload.enabled: false

# To enable hints based autodiscover, remove `filebeat.config.inputs` configuration and uncomment this:
#filebeat.autodiscover:
# providers:
# - type: kubernetes
# hints.enabled: true

processors:
- add_cloud_metadata:

Expand Down
6 changes: 6 additions & 0 deletions deploy/kubernetes/filebeat/filebeat-configmap.yaml
View file
Open in desktop
Expand Up @@ -19,6 +19,12 @@ data:
# Reload module configs as they change:
reload.enabled: false

# To enable hints based autodiscover, remove `filebeat.config.inputs` configuration and uncomment this:
#filebeat.autodiscover:
# providers:
# - type: kubernetes
# hints.enabled: true

processors:
- add_cloud_metadata:

Expand Down
41 changes: 38 additions & 3 deletions deploy/kubernetes/metricbeat-kubernetes.yaml
View file
Open in desktop
Expand Up @@ -2,7 +2,7 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: metricbeat-config
name: metricbeat-daemonset-config
namespace: kube-system
labels:
k8s-app: metricbeat
Expand All @@ -14,6 +14,13 @@ data:
# Reload module configs as they change:
reload.enabled: false

# To enable hints based autodiscover uncomment this:
#metricbeat.autodiscover:
# providers:
# - type: kubernetes
# host: ${HOSTNAME}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is $HOSTNAME always set? and is it always set to a hostname in the list of nodes? (e.g. I think nodes can be registered with their IPs)

Alternatively we can add the NODE_NAME environment variable.

        - name: NODE_NAME
          valueFrom:
            fieldRef:
              fieldPath: spec.nodeName

# hints.enabled: true

processors:
- add_cloud_metadata:

Expand Down Expand Up @@ -68,6 +75,7 @@ data:
- container
- volume
period: 10s
host: ${HOSTNAME}
hosts: ["localhost:10255"]
---
# Deploy a Metricbeat instance per node for node metrics retrieval
Expand Down Expand Up @@ -146,7 +154,7 @@ spec:
- name: config
configMap:
defaultMode: 0600
name: metricbeat-config
name: metricbeat-daemonset-config
- name: modules
configMap:
defaultMode: 0600
Expand All @@ -158,6 +166,32 @@ spec:
---
apiVersion: v1
kind: ConfigMap
metadata:
name: metricbeat-deployment-config
namespace: kube-system
labels:
k8s-app: metricbeat
data:
metricbeat.yml: |-
metricbeat.config.modules:
# Mounted `metricbeat-daemonset-modules` configmap:
path: ${path.config}/modules.d/*.yml
# Reload module configs as they change:
reload.enabled: false

processors:
- add_cloud_metadata:

cloud.id: ${ELASTIC_CLOUD_ID}
cloud.auth: ${ELASTIC_CLOUD_AUTH}

output.elasticsearch:
hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
username: ${ELASTICSEARCH_USERNAME}
password: ${ELASTICSEARCH_PASSWORD}
---
apiVersion: v1
kind: ConfigMap
metadata:
name: metricbeat-deployment-modules
namespace: kube-system
Expand All @@ -176,6 +210,7 @@ data:
# Uncomment this to get k8s events:
#- event
period: 10s
host: ${HOSTNAME}
hosts: ["kube-state-metrics:8080"]
---
# Deploy singleton instance in the whole cluster for some unique data sources, like kube-state-metrics
Expand Down Expand Up @@ -235,7 +270,7 @@ spec:
- name: config
configMap:
defaultMode: 0600
name: metricbeat-config
name: metricbeat-deployment-config
- name: modules
configMap:
defaultMode: 0600
Expand Down
10 changes: 9 additions & 1 deletion deploy/kubernetes/metricbeat/metricbeat-daemonset-configmap.yaml
View file
Open in desktop
Expand Up @@ -2,7 +2,7 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: metricbeat-config
name: metricbeat-daemonset-config
namespace: kube-system
labels:
k8s-app: metricbeat
Expand All @@ -14,6 +14,13 @@ data:
# Reload module configs as they change:
reload.enabled: false

# To enable hints based autodiscover uncomment this:
#metricbeat.autodiscover:
# providers:
# - type: kubernetes
# host: ${HOSTNAME}
# hints.enabled: true

processors:
- add_cloud_metadata:

Expand Down Expand Up @@ -68,4 +75,5 @@ data:
- container
- volume
period: 10s
host: ${HOSTNAME}
hosts: ["localhost:10255"]
2 changes: 1 addition & 1 deletion deploy/kubernetes/metricbeat/metricbeat-daemonset.yaml
View file
Open in desktop
Expand Up @@ -74,7 +74,7 @@ spec:
- name: config
configMap:
defaultMode: 0600
name: metricbeat-config
name: metricbeat-daemonset-config
- name: modules
configMap:
defaultMode: 0600
Expand Down
27 changes: 27 additions & 0 deletions deploy/kubernetes/metricbeat/metricbeat-deployment-configmap.yaml
View file
Open in desktop
@@ -1,5 +1,31 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: metricbeat-deployment-config
namespace: kube-system
labels:
k8s-app: metricbeat
data:
metricbeat.yml: |-
metricbeat.config.modules:
# Mounted `metricbeat-daemonset-modules` configmap:
path: ${path.config}/modules.d/*.yml
# Reload module configs as they change:
reload.enabled: false

processors:
- add_cloud_metadata:

cloud.id: ${ELASTIC_CLOUD_ID}
cloud.auth: ${ELASTIC_CLOUD_AUTH}

output.elasticsearch:
hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
username: ${ELASTICSEARCH_USERNAME}
password: ${ELASTICSEARCH_PASSWORD}
---
apiVersion: v1
kind: ConfigMap
metadata:
name: metricbeat-deployment-modules
namespace: kube-system
Expand All @@ -18,4 +44,5 @@ data:
# Uncomment this to get k8s events:
#- event
period: 10s
host: ${HOSTNAME}
hosts: ["kube-state-metrics:8080"]
2 changes: 1 addition & 1 deletion deploy/kubernetes/metricbeat/metricbeat-deployment.yaml
View file
Open in desktop
Expand Up @@ -55,7 +55,7 @@ spec:
- name: config
configMap:
defaultMode: 0600
name: metricbeat-config
name: metricbeat-deployment-config
- name: modules
configMap:
defaultMode: 0600
Expand Down