SpanLimits going over limit set for max events per spans

[wperron]

I'm trying to set the limit on events per spans to 0, but events are still being exported, bypassing the limit set in the config. Here's a minimal repro:

/// Run with `OTEL_SPAN_EVENT_COUNT_LIMIT=0 cargo run`

use opentelemetry::{
    global,
    sdk::{self, export::trace::stdout},
    trace::{Span, Tracer},
};

fn main() {
    // Create a new trace pipeline that prints to stdout
    let tracer = stdout::new_pipeline()
        .with_trace_config(sdk::trace::config())
        .install_simple();

    {
        let mut span = tracer.start("doing work");
        span.add_event("my event", vec![]);
        span.add_event("my other event", vec![]);
    }

    // Shutdown trace pipeline
    global::shutdown_tracer_provider();
}

Outputs:

SpanData { span_context: SpanContext { trace_id: 681f5965702c75e41724caf5b33b9d1b, span_id: f682e85763f26d16, trace_flags: TraceFlags(1), is_remote: false, trace_state: TraceState(None) }, parent_span_id: 0000000000000000, span_kind: Internal, name: "doing work", start_time: SystemTime { tv_sec: 1689254989, tv_nsec: 766169000 }, end_time: SystemTime { tv_sec: 1689254989, tv_nsec: 766173000 }, attributes: EvictedHashMap { map: {}, evict_list: [], max_len: 128, dropped_count: 0 }, events: EvictedQueue { queue: Some([Event { name: "my event", timestamp: SystemTime { tv_sec: 1689254989, tv_nsec: 766172000 }, attributes: [KeyValue { key: Static("my_attribute"), value: String(Static("some_value")) }], dropped_attributes_count: 0 }]), max_len: 0, dropped_count: 1 }, links: EvictedQueue { queue: None, max_len: 128, dropped_count: 0 }, status: Unset, resource: Resource { attrs: {Static("service.name"): String(Owned("unknown_service"))}, schema_url: None }, instrumentation_lib: InstrumentationLibrary { name: "opentelemetry", version: Some("0.19.0"), schema_url: None } }

I thought this might be an edge case for 0 and tried setting the limit to 1 but the same issue appears. I also tried overwriting the limit in code rather than through the environment variable, no luck.

I haven't looked to see if this also applied to the other limits as well.

---

Edit: This also has a negative side effect when setting events with SpanBuilder, if the limit is set to 0 in that case, all the events are attached to the EvictedQueue, basically putting no limit on the number of events or links on a span.

let sb = tracer.span_builder("doing work").with_events(vec![
    Event::new("my event", SystemTime::now(), vec![], 0),
    Event::new("my second event", SystemTime::now(), vec![], 0),
    Event::new("my third event", SystemTime::now(), vec![], 0),
]);
let _ = tracer.build_with_context(sb, &Context::current());

This particularly affects tracing-opentelemetry since that's how they create OpenTelemetry spans from their spans.

[wperron]

Just verified all the limit options:

• ✅ max_attributes_per_event
• ✅ max_attributes_per_link
• ✅ max_attributes_per_span
• 🚫 max_events_per_span
• 🚫 max_links_per_span

[wperron]

Investigating this further, I think this is a bug with EvictedQueue -- all the max_attributes_* limit use an EvictedHashMap and don't exhibit the issue, while max_events_per_span and max_links_per_span use EvictedQueue and do exhibit the issue.

[cijothomas]

@wperron Can you describe more what is the use case for setting the limit to 0 for span events? It is mentioned this is needed for tracing-opentelemetry - but not clear.. If the intention is not to send tracing events as Otel Span Events, then tracing-opentelemetry can provide a bool flag to control that, right?

If you can comment on #1283 (comment), that'd help while working on some improvements (including potentially removing this feature) in the area!