Forward logs to different external system(s) by mburke5678 · Pull Request #24029 · openshift/openshift-docs

mburke5678 · 2020-07-21T20:17:13Z

https://issues.redhat.com/browse/LOG-561
https://issues.redhat.com/browse/OSDOCS-1074
Alan note; https://github.com/alanconway/cluster-logging-operator/blob/clf-url-parsing/docs/output_url_field.md#L1

openshift-docs-preview-bot · 2020-07-21T21:04:26Z

The preview will be available shortly at:

https://logging-log-forward-ga--ocpdocs.netlify.com/

modules/cluster-logging-log-forwarding-about.adoc

modules/cluster-logging-collector-log-forward-examples.adoc

StevenBarre · 2020-07-24T16:28:09Z

modules/cluster-logging-log-forwarding-about.adoc

I think this sentence needs finishing?

mburke5678 · 2020-08-11T21:38:55Z

Logging team review at: https://docs.google.com/document/d/1ORMlZH_5RPRz1G0kFlYsFFT3ANmHtcj7JTz8BFV4R-w/edit?ts=5f32f370#

QiaolingTang · 2020-08-21T03:33:08Z

modules/cluster-logging-collector-log-forward-configure.adoc

@mburke5678 could you please remove this output? In clusterlogforwarder, we have a default output named as default, if user want to set forward logs to default ES, just need to add - default to the outputRefs, no need to define an output. You can find the details in https://issues.redhat.com/browse/LOG-706.

Removed this file.

QiaolingTang · 2020-08-21T05:44:36Z

modules/cluster-logging-collector-log-forward-es.adoc

Start from 4.5, we use ES v6.x, so I think maybe we should say you can optionally forward logs to an external Elasticsearch v6.x instance.

@sichvoge What did we decide on this?

QiaolingTang · 2020-08-21T06:09:41Z

modules/cluster-logging-collector-log-forward-es.adoc

apiVersion: logging.openshift.io/v1

gkarager · 2020-08-21T06:16:15Z

modules/cluster-logging-collector-legacy.adoc

@mburke5678 could you please update this part from line 57 to 88?
With the recent changes syslog supports TLS and both RFC 3164 & 5424.
For ref: https://github.com/openshift/cluster-logging-operator/blob/master/manifests/4.6/logging.openshift.io_clusterlogforwarders_crd.yaml#L152

QiaolingTang · 2020-08-21T06:28:37Z

modules/cluster-logging-collector-log-forward-es.adoc

apiVersion: logging.openshift.io/v1 kind: ClusterLogForwarder

QiaolingTang · 2020-08-21T06:29:17Z

modules/cluster-logging-collector-log-forward-es.adoc

url: http://elasticsearch.example.com:9200

QiaolingTang · 2020-08-21T06:31:37Z

modules/cluster-logging-collector-log-forward-es.adoc

container-logs is not accurate, I was thinking we could use another name.

QiaolingTang · 2020-08-21T06:32:34Z

modules/cluster-logging-collector-log-forward-examples.adoc

apiVersion: logging.openshift.io/v1 kind: ClusterLogForwarder

Removed this topic.

QiaolingTang · 2020-08-21T06:33:04Z

modules/cluster-logging-collector-log-forward-examples.adoc

url: http://elasticsearch.example.com:9200

Removed this topic.

QiaolingTang · 2020-08-21T06:33:41Z

modules/cluster-logging-collector-log-forward-examples.adoc

apiVersion: logging.openshift.io/v1 kind: ClusterLogForwarder

Removed this topic.

QiaolingTang · 2020-08-21T06:34:06Z

modules/cluster-logging-collector-log-forward-examples.adoc

url: http://elasticsearch.example.com:9200

Removed this topic.

QiaolingTang · 2020-08-21T06:34:48Z

modules/cluster-logging-collector-log-forward-examples.adoc

url: 'tcp://fluentdserver.example.com:24224'

Removed this topic.

QiaolingTang · 2020-08-21T06:38:59Z

modules/cluster-logging-collector-log-forward-examples.adoc

url: http://elasticsearch.insecure.example.com.svc:9200

@mburke5678 out of curiosity, why do you give so many outputs here?

Removed this topic.

QiaolingTang · 2020-08-21T06:46:22Z

modules/cluster-logging-collector-log-forward-examples.adoc

Per my understanding, no matter what kind of authentication the forward use, if user want to enable TLS, then they must create a secret and specify it in the cluster log forwarder CR. @alanconway could you please help to confirm this?

Removed this topic.

QiaolingTang · 2020-08-21T06:48:56Z

modules/cluster-logging-collector-log-forward-examples.adoc

apiVersion: logging.openshift.io/v1 kind: ClusterLogForwarder

Removed this topic.

QiaolingTang · 2020-08-21T06:49:14Z

modules/cluster-logging-collector-log-forward-examples.adoc

url: https://elasticsearch.example.com:9200

Removed this topic.

alanconway

Yep!

anpingli

/lgtm

anpingli · 2020-09-09T08:22:22Z

modules/cluster-logging-collector-legacy.adoc

+[id="cluster-logging-collector-legacy_{context}"]
+= Forwarding logs using the legacy configuration method
+
+You can configure cluster logging to send logs to destinations outside of your {product-title} cluster instead of the default Elasticsearch log store using Fluentd or syslog by creating a configuration file and ConfigMap.


@mburke5678 Yes, unmanaged is not required

openshift-ci-robot · 2020-09-14T19:04:46Z

New changes are detected. LGTM label has been removed.

mburke5678 · 2020-09-25T15:51:43Z

http://file.rdu.redhat.com/~mburke/logging-log-forward-ga/logging/cluster-logging-external.html

sheriff-rh

LGTM! Great work! 🎉 🌮 🥇

mburke5678 · 2020-09-29T20:04:06Z

I am merging this PR as #25229 depends on it. Any further comments can be tracked in a new PR.

openshift-ci-robot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Jul 21, 2020

mburke5678 mentioned this pull request Jul 21, 2020

4.5.x Changes to log forward API for 4.5 #22656

Closed

mburke5678 added branch/enterprise-4.5 branch/enterprise-4.6 labels Jul 21, 2020

mburke5678 added this to the Future Release milestone Jul 21, 2020

openshift-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Jul 23, 2020

StevenBarre reviewed Jul 24, 2020

View reviewed changes

mburke5678 force-pushed the logging-log-forward-ga branch from dc067c1 to 30eb7df Compare July 30, 2020 15:37

openshift-ci-robot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Aug 5, 2020

mburke5678 force-pushed the logging-log-forward-ga branch from bb3c1c2 to 7ca6a28 Compare August 8, 2020 01:01

QiaolingTang suggested changes Aug 21, 2020

View reviewed changes

QiaolingTang reviewed Aug 21, 2020

View reviewed changes

modules/cluster-logging-collector-log-forward-es.adoc Outdated

Copy link

QiaolingTang Aug 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

apiVersion: logging.openshift.io/v1

gkarager suggested changes Aug 21, 2020

View reviewed changes

QiaolingTang reviewed Aug 21, 2020

View reviewed changes

modules/cluster-logging-collector-log-forward-es.adoc Outdated

Copy link

QiaolingTang Aug 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

url: http://elasticsearch.example.com:9200

QiaolingTang reviewed Aug 21, 2020

View reviewed changes

modules/cluster-logging-collector-log-forward-es.adoc Outdated

Copy link

QiaolingTang Aug 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

container-logs is not accurate, I was thinking we could use another name.

QiaolingTang reviewed Aug 21, 2020

View reviewed changes

mburke5678 force-pushed the logging-log-forward-ga branch from 2356ac4 to c35b5e6 Compare September 2, 2020 18:26

alanconway approved these changes Sep 2, 2020

View reviewed changes

anpingli reviewed Sep 9, 2020

View reviewed changes

openshift-ci-robot assigned anpingli Sep 9, 2020

openshift-ci-robot added the lgtm Indicates that a PR is ready to be merged. label Sep 9, 2020

mburke5678 mentioned this pull request Sep 14, 2020

4.6 After installation infra and audit index pattern not available in Kibana #25456

Merged

openshift-ci-robot removed the lgtm Indicates that a PR is ready to be merged. label Sep 14, 2020

openshift-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 14, 2020

mburke5678 mentioned this pull request Sep 21, 2020

[WIP] 4.6 Logging cluster labels #25229

Closed

mburke5678 force-pushed the logging-log-forward-ga branch from 3a7357f to 1044e8c Compare September 24, 2020 20:16

openshift-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Sep 24, 2020

mburke5678 force-pushed the logging-log-forward-ga branch 4 times, most recently from b9cd829 to 366befc Compare September 25, 2020 15:49

Forward logs to different external system(s)

92f1113

mburke5678 force-pushed the logging-log-forward-ga branch from 366befc to 92f1113 Compare September 26, 2020 01:06

sheriff-rh approved these changes Sep 29, 2020

View reviewed changes

sheriff-rh added the peer-review-done Signifies that the peer review team has reviewed this PR label Sep 29, 2020

mburke5678 removed the branch/enterprise-4.5 label Sep 29, 2020

mburke5678 merged commit f3ef34d into openshift:master Sep 29, 2020

mburke5678 mentioned this pull request Sep 29, 2020

[enterprise-4.6] Forward logs to different external system(s) #25871

Merged

openshift deleted a comment from openshift-cherrypick-robot Sep 29, 2020

mburke5678 mentioned this pull request Sep 29, 2020

Logging cluster labels #25882

Merged

mburke5678 deleted the logging-log-forward-ga branch October 21, 2020 17:46

vikram-redhat modified the milestones: Future Release, OCP 4.6 GA Oct 27, 2020

Conversation

mburke5678 commented Jul 21, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

openshift-docs-preview-bot commented Jul 21, 2020

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mburke5678 commented Aug 11, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

QiaolingTang Aug 21, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

alanconway left a comment

Choose a reason for hiding this comment

Uh oh!

anpingli left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

openshift-ci-robot commented Sep 14, 2020

Uh oh!

mburke5678 commented Sep 25, 2020

Uh oh!

mburke5678 commented Jul 21, 2020 •

edited

Loading

QiaolingTang Aug 21, 2020 •

edited

Loading