Skip to content

Latest commit

 

History

History
85 lines (75 loc) · 5.03 KB

readme.md

File metadata and controls

85 lines (75 loc) · 5.03 KB
Raw

OpenShift EFK - external setup

This repo describes available options and required steps for connecting OpenShift EFK stack to external ES and Kibana instances.

Option 1 - Connect FluentD instances to external stand alone (not OCP based) ES and Kibana setup

  • Download and extract ES and Kibana ES: https://www.elastic.co/downloads/past-releases/elasticsearch-5-6-13 Kibana: https://www.elastic.co/downloads/past-releases/kibana-5-6-13

  • Extract the archives and start ES & Kibana cd elasticsearch && ./bin/elasticsearch cd kibana && ./bin/kibana

  • Import Indexes and Indexes patters into ES instance

    • The es-imoprter.py script import indexes and indexes pattern into external EFK setup. The import script uses following JSON source files
    • index_templates.json
    • index_pattern.json
    • index_patterns_names.json
    Usage

    python es-importer.py [ES_HOST_IP] [ES_PORT]

    Data source

    The indexes and indexes patters was exported directly from running ES pod. Execute following command to re-export the data again

    • Export index templates: curl -s -X GET --cacert /etc/elasticsearch/secret/admin-ca --cert /etc/elasticsearch/secret/admin-cert --key /etc/elasticsearch/secret/admin-key https://localhost:9200/_template
    • Export index patterns: The index patterns template could be found official GitHub repo, here
    • Export index patterns names: Could be either from running Kibana UI or by parsing the index_templates files. As from now we have following patterns names: [".operations.*", ".all",".orphaned.*","project.*"]

  • Install EFK on OCP

    cd /usr/share/ansible/openshift-ansible
ansible-playbook playbooks/openshift-logging/config.yml

  • Update FluentD daemonet with oc edit daemonsets logging-fluentd and set ES_HOST, ES_PORT, OPS_HOST, and OPS_PORT to point to IP of the external ES instance. More details here

  • Update FluentD daemonet with oc edit daemonsets logging-fluentd and set image to the following value: image: docker.io/dimssss/ose-logging-fluentd:v3.11.83. To build your own FluentD image, take a look on custom-fluetd folder in that repo.

Option 2 - Connect FluentD instances to external EFK which is running inside other OpenShift cluster

  • Deploy EFK on both OpenShift clusters. (as for example I'll call it external and internal, internal OCP will exports logs to EFK which is runs on external OCP) by running following command.

    cd /usr/share/ansible/openshift-ansible
ansible-playbook playbooks/openshift-logging/config.yml

  • Make sure everything is up and running

    userx:~ λ oc get pods -n openshift-logging
NAME                                      READY     STATUS    RESTARTS   AGE
logging-es-data-master-2nghl23v-1-77bbm   2/2       Running   0          20m
logging-fluentd-9nztq                     1/1       Running   0          22m
logging-fluentd-hn94z                     1/1       Running   1          2d
logging-fluentd-n2tsz                     1/1       Running   1          2d
logging-fluentd-s9rtg                     1/1       Running   1          2d
logging-kibana-ops-1-kxzcx                2/2       Running   4          3d

  • Export FluentD certificates from external cluster and use them in internal cluster Export from external: oc get secrets logging-fluentd -o yaml -n openshift-logging Edit logging-fluentd secret in internal cluster and update the ca cert key and ops-ca ops-cert ops-key with the values from external cluster.

  • To allow FluentD instances to connect to external cluster expose ES instance of external cluster by creating a route

    apiVersion: route.openshift.io/v1
kind: Route
metadata:
  name: logging-es
  namespace: openshift-logging
spec:
  host: logging-es-openshift-logging.router.default.svc.cluster.local
  tls:
    termination: passthrough
  to:
    kind: Service
    name: logging-es
    weight: 100

  • Test the exposed route by running following command: curl -k -s -X GET --cacert admin-ca --cert admin-cert --key admin-key https://logging-es-openshift-logging.router.default.svc.cluster.local/_template the admin-ca admin-cert and admin-key could be retrieved from logging-elasticsearch secret.

  • Update FluentD daemonet with oc edit daemonsets logging-fluentd and set ES_HOST, ES_PORT, OPS_HOST, and OPS_PORT more details here

  • Open Kibana dashboard and test the results