Skip to content

Errors in Loki logs #1014

Description

@Enkidu93

Greg has been seeing a lot of these errors in the logs - apparently for a very long time. They aren't causing problems, but he's trying to clean things up a bit so that he can actually see true errors when they occur. I believe they ultimately have something to do with trying to parse the logs with a regex which fails to match.

From Greg: Here's a sample of two errors that I'm seeing in the loki logs in both the nlp and serval namespace that I believe is coming from the flow/outputs:

2026-07-02 19:09:28 +0000 [warn]: #0 send an error event to @ERROR: error_class=ArgumentError error="log does not exist" location="/usr/local/bundle/gems/fluentd-1.18.0/lib/fluent/plugin/filter_parser.rb:66:in `block in filter_stream'" tag="kubernetes.var.log.containers.serval-api-79f4979ff6-xfpkd_serval_serval-api-c1e989d56198e99edbf45f27876760cc23a45965923204b8dffa9ce685aece82.log" time=2026-07-02 19:09:28.746553110 +0000

2026-07-02 19:10:28 +0000 [warn]: #0 send an error event to @ERROR: error_class=Fluent::Plugin::ConcatFilter::TimeoutError error="Timeout flush: kubernetes.var.log.containers.serval-api-79f4979ff6-xfpkd_serval_serval-api-c1e989d56198e99edbf45f27876760cc23a45965923204b8dffa9ce685aece82.log:default" location=nil tag="kubernetes.var.log.containers.serval-api-79f4979ff6-xfpkd_serval_serval-api-c1e989d56198e99edbf45f27876760cc23a45965923204b8dffa9ce685aece82.log" time=2026-07-02 19:10:28.071596998 +0000

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

Projects

Status
🔖 Ready

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions