collection.adoc

Collection and Log Forwarding Features

Administrators create a ClusterLogForwarder instance to specify which logs will be collected, how they will be transformed, and to where they will be forwarded. The operator will deploy a collector based upon this specification. There are two collector options supported by this operator: fluentd and vector. Vector is the newest collector implementation and the difference in feature parity is identified below.

Note	Some features implemented in fluentd may not be implemented for vector. Justifications for this variance is documented below.

See the ClusterLogForwarder documentation for deploying and administration information.

Log Sources

Following is a list of possible log sources that are collected from each node in the cluster:

Table 1. Log Sources

Feature	Desc.	Fluentd	Vector
App container logs	Logs generated by container workloads in non-infrastructure namespaces	✓	✓
Application label selector	Selectively collect application by namespace or pod label selector	✓	✓
Container log selection using Kubernetes pod metadata	Enhancement of application label selectors to choose inputs using additional metadata		✓
Infra container logs	Logs generated by container workloads in infrastructure namespaces	✓	✓
Infra journal logs	Logs generated by node services from the nodes' journald service	✓	✓
Individual infra log sources	Explicit selection of journal and/or container logs		✓
Kubernetes api audit logs	Kubernetes api service logs	✓	✓
OpenShift api audit logs	OpenShift api service logs	✓	✓
OVN audit logs	Open Virtual Network Logs written to the node filesystem	✓	✓
Auditd logs	Linux auditd logs written to the node filesystem	✓	✓
Individual audit log sources	Explicit selection of audit log sources		✓

Outputs

Table 2. Output Destinations

Feature	Protocol	Tested with	Fluentd	Vector
Azure Monitor Logs				✓
Cloudwatch	REST over HTTPS		✓	✓
Elasticsearch		v6.8.1 v7.12.2 v8.6.1		✓
Google Cloud Logging				✓
Kafka	kafka 0.11	kafka 2.4.1 kafka 2.7.0	✓	✓
Fluent Forward	fluentd forward v1	fluentd 1.14.6 logstash 7.10.1	✓
Loki	REST over HTTP(S)	Loki 2.3.0	✓	✓
Splunk	HEC	v9.0.0		✓
Syslog	RFC3164,RFC5424	rsyslog 8.39.0	✓	✓
Http	Http 1.1	vector 0.28-1 vector 0.34-1 fluentd 1.14.6 fluentd 1.16.2		✓

Authorization and Authentication

Table 3. Authorization and Authentication Methods

Feature	Output Type	Fluentd	Vector
Token	loki, splunk, http
Certificates	elasticsearch, kafka, fluentd forward, splunk, http	✓	✓
Cloud service keys	cloudwatch	✓	✓
Cloud service keys	google		✓
Username / Password	elasticsearch, kafka, http
Security Token Service (STS)	cloudwatch	✓	✓
SASL	kafka	✓	✓

Normalizations and Transformations

Table 4. Normalizations and Transformations

Feature	Desc.	Fluentd	Vector
Viaq data model	See reference document for details. Not all fields are supported for both collector implementations	✓	✓
Loglevel		✓	✓
JSON Parsing		✓	✓
Structured Index for Elasticsearch JSON parsing		✓	✓
Multiline error detection	See feature document for languages supported by each collector	✓	✓
Split indices for multi-container pods		✓	✓
Static labels for forwarding pipelines		✓	✓
Drop Filter			✓
Prune Filter			✓

Security and Compliance

Table 5. Security and Compliance

Feature	Desc.	Fluentd	Vector
FIPS	Tested on a FIPS enabled cluster	✓	✓
Crypto Export		?	?
TLS Security Profile Compliance	Comply with OCP cluster-wide cryptographic profiles for internal communication and allow configuration of outbound connection profiles. See details	n/a	✓

Tuning

Vector Output Tuning

Following is a list of output tuning options based upon the enhancement document. Not all outputs support all tuning options.

Parameter	Desc.
Delivery	The mode for log forwarding. AtLeastOnce (default): The forwarder will block in an attempt to deliver all messages. When the tuning spec is added to an output, this additionally configures an internal, durable buffer so the collector can attempt to forward any logs read before it restarted AtMostOnce: The forwarder may provide better throughput but also may drop logs in the event of spikes in volume and backpressure from the output. Undelivered, collected logs will be lost on collector restart. NOTE:: Log collection and forwarding is best effort. AtLeastOnce delivery mode does not guarantee logs will not be lost.
Compression	The compression algorithm to use to compress the data before sending over the network. gzip none snappy zlib zstd lz4 NOTE: An output type may not support all available compression options or compression.
MaxWrite	The resource quantity that limits the maximum payload of a single "send" to the output.
MinRetryDuration	The minimum time to wait between attempts to retry after a delivery failure.
MaxRetryDuration	The maximum time to wait between retry attempts after a delivery failure.

Table 6. Fluentd Tuning

Feature	Desc.
Source	readLinesLimit
Output Buffering	chunklimitsize totallimitsize overflowaction flushthreadcount flushmode flushinterval retrywait retrytype retrymaxinterval retrytimeout

Metrics and Alerting

Table 7. Metrics and Alerting

Feature	Desc.	Fluentd	Vector
Logs collected		✓
Container logs generated		✓	✓
Collector dashboard		✓	✓
Collector alerts		✓

Miscellaneous

Table 8. Miscellaneous

Feature	Desc.	Fluentd	Vector
Global Proxy		✓	✓
Architecture
…​x86		✓	✓
…​ARM		✓	✓
…​Power PC		✓	✓
…​IBM Z		✓	✓
IPv6		✓	✓