Open
Description
Component(s)
receiver/filelog, pkg/stanza/fileconsumer
What happened?
Description
Tried setting up OpenObserve, so tried installing an opentelemetry collector to collect logs and system info and etc.
The default otel-config.yaml from openobserve tries has a filelog receiver with a include: [ /var/log/**log ] wildcard to match all log files in /var/log/.
If there exists a log file that can't be opened the collector will retry every second or so, spamming logs in the process, never stopping or skipping the file.
For every fail it writes a big block of text into the system log.
This is how permissions look on the log file:
-rw-r----- 1 root adm 145009 Apr 15 21:58 /var/log/cloud-init-output.log
Steps to Reproduce
- Create a file it can't read
- Configure filelog to read said file
- Start the collector
- Watch the never-ending log spam
Expected Result
One log line informing me a file can't be open.
Actual Result
Log spam, see attached logs.
Collector version
v0.124.1
Environment information
Environment
OS: Debian GNU/Linux 12 (bookworm) cloud image
Compiler(if manually compiled): N/A
OpenTelemetry Collector configuration
receivers:
journald:
directory: /var/log/journal
filelog/std:
include: [ /var/log/**log ]
# start_at: beginning
hostmetrics:
root_path: /
collection_interval: 30s
scrapers:
cpu:
disk:
filesystem:
load:
memory:
network:
paging:
processes:
# process: # a bug in the process scraper causes the collector to throw errors so disabling it for now
processors:
resourcedetection/system:
detectors: ["system"]
system:
hostname_sources: ["os"]
memory_limiter:
check_interval: 1s
limit_percentage: 75
spike_limit_percentage: 15
batch:
send_batch_size: 10000
timeout: 10s
extensions:
zpages: {}
exporters:
otlphttp/openobserve:
endpoint: http://monitoring.denys.home:5080/api/default/
headers:
Authorization: "XXX"
otlphttp/openobserve_journald:
endpoint: http://monitoring.denys.home:5080/api/default/
headers:
Authorization: "XXX"
stream-name: journald
service:
extensions: [zpages]
pipelines:
metrics:
receivers: [hostmetrics]
processors: [resourcedetection/system, memory_limiter, batch]
exporters: [otlphttp/openobserve]
logs:
receivers: [filelog/std]
processors: [resourcedetection/system, memory_limiter, batch]
exporters: [otlphttp/openobserve]
logs/journald:
receivers: [journald]
processors: [resourcedetection/system, memory_limiter, batch]
exporters: [otlphttp/openobserve_journald]Log output
Apr 18 16:00:50 monitoring otelcol-contrib[136903]: 2025-04-18T16:00:50.717+0800 error fileconsumer/file.go:181 Failed to open file {"component": "fileconsumer", "error": "open /var/log/cloud-init-output.log: permission denied"}
Apr 18 16:00:50 monitoring otelcol-contrib[136903]: github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/fileconsumer.(*Manager).makeFingerprint
Apr 18 16:00:50 monitoring otelcol-contrib[136903]: github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.124.1/fileconsumer/file.go:181
Apr 18 16:00:50 monitoring otelcol-contrib[136903]: github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/fileconsumer.(*Manager).makeReaders
Apr 18 16:00:50 monitoring otelcol-contrib[136903]: github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.124.1/fileconsumer/file.go:208
Apr 18 16:00:50 monitoring otelcol-contrib[136903]: github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/fileconsumer.(*Manager).consume
Apr 18 16:00:50 monitoring otelcol-contrib[136903]: github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.124.1/fileconsumer/file.go:158
Apr 18 16:00:50 monitoring otelcol-contrib[136903]: github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/fileconsumer.(*Manager).poll
Apr 18 16:00:50 monitoring otelcol-contrib[136903]: github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.124.1/fileconsumer/file.go:140
Apr 18 16:00:50 monitoring otelcol-contrib[136903]: github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza/fileconsumer.(*Manager).startPoller.func1
Apr 18 16:00:50 monitoring otelcol-contrib[136903]: github.com/open-telemetry/opentelemetry-collector-contrib/pkg/stanza@v0.124.1/fileconsumer/file.go:110Additional context
No response