Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
65 changes: 65 additions & 0 deletions modules/otel-collecting-container-logs-with-filelog-sidecar.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
// Module included in the following assemblies:
//
// * observability/otel/otel-sending-traces-logs-and-metrics-to-otel-collector.adoc

:_mod-docs-content-type: PROCEDURE
[id="collecting-container-logs-with-filelog-sidecar_{context}"]
= Collecting container logs with the Filelog Receiver and sidecar injection

Collect logs from application containers by using the Filelog Receiver with sidecar injection. This approach is useful when your application writes logs to files within the container filesystem.

The {OTELOperator} allows to inject an OpenTelemetry Collector as a sidecar container into a application pod. While to access generated files both pods need to share a volume between
the application container and the sidecar collector. Afterwards the Filelog Receiver can tail log files and apply operators to parse and transform the logs.

.Prerequisites

* You have access to the cluster as a cluster administrator with the `cluster-admin` role.

. Create an OpenTelemetry Collector instance configured with the filelog Receiver in sidecar mode. Ensure a matching `volumeMounts` entry is set to grant the sidecar access to your targeted files.
+
[source,yaml]
----
apiVersion: opentelemetry.io/v1beta1
kind: OpenTelemetryCollector
metadata:
name: filelog
namespace: otel-logging
spec:
mode: sidecar
volumeMounts: # <1>
- name: logs
mountPath: /var/log/app
config:
receivers:
filelog:
include: # <2>
- /var/log/app/*.log
operators:
- type: regex_parser
regex: '^(?P<timestamp>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) \[(?P<level>\w+)\] (?P<message>.*)$'
timestamp:
parse_from: attributes.timestamp
layout: '%Y-%m-%d %H:%M:%S'
processors: {}
exporters:
debug:
verbosity: detailed
service:
pipelines:
logs:
receivers: [filelog]
processors: []
exporters: [debug]
----
<1> Defines the volume mount that the sidecar collector will use to access the log files. This volume must match the volume name defined in the application deployment.
<2> File glob patterns matching the log files to tail. The Filelog Receiver watches these paths for new log entries.
+
[IMPORTANT]
====
The `volumeMounts` field in the `OpenTelemetryCollector` custom resource is critical for the sidecar to access log files. The volume specified here must be defined in the application's `Deployment` or `Pod` specification, and both the application container and the sidecar collector must mount the same volume.
====

[role="_additional-resources"]
.Additional resources
* xref:../../../observability/otel/otel-collector/otel-collector-receivers.adoc#filelog-receiver_otel-collector-receivers[Filelog Receiver]
* xref:../../../observability/otel/otel-collector/otel-collector-receivers.adoc#otel-collector-receivers[Receivers]
Original file line number Diff line number Diff line change
Expand Up @@ -650,6 +650,14 @@ include::snippets/technology-preview.adoc[]
<1> A list of file glob patterns that match the file paths to be read.
<2> An array of Operators. Each Operator performs a simple task such as parsing a timestamp or JSON. To process logs into a desired format, chain the Operators together.

[NOTE]
====
When using the Filelog Receiver in sidecar mode to collect logs from application containers, you must configure volume mounts in
the `OpenTelemetryCollector` custom resource. The collector needs access to the log files through a shared volume such
as `emptyDir` that is mounted in both the application container and the sidecar collector container. For a complete example,
see "Collecting container logs with the Filelog Receiver and sidecar injection".
====

[id="journald-receiver_{context}"]
== Journald Receiver

Expand Down