Skip to content

[otel log] Handle logs without Timestamp field better. #5311

@c0per

Description

@c0per

Describe the bug

Currently, quickwit is ignoring otel logs with a null (or 0) Timestamp.

// quickwit-opentelemetry::otlp::logs.rs line 441
for log_record in scope_log.log_records {
    if log_record.time_unix_nano == 0 {
        rate_limited_error!(limit_per_min = 10, "skipping record");
        num_parse_errors += 1;
        continue;
    }
    // ...
}

Some instrumentation only set ObservedTimestamp and leave Timestamp null.
Maybe we should default to ObservedTimestamp when Timestamp is not set.

Steps to reproduce
Use any Otel logs exporter which doesn't set the timestamp field.
In my case, I'm using rust tracing + opentelemetry-appender-tracing.

Expected behavior

Other people have encountered this problem using the same setup with the official otel collector (open-telemetry/opentelemetry-collector-contrib#11752 (comment)).

The official otel collector exporter use ObservedTimestamp when Timestamp is not set.
The PR: open-telemetry/opentelemetry-collector-contrib#31844

Maybe Quickwit should also do this. Default to ObservedTimestamp when Timestamp is not set, instead of throwing away logs without a timestamp.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions