Skip to content

fix: test_tracing_backend.py tests always skip (Telemetry not initialized) #754

@planetf1

Description

@planetf1

Problem

All 6 tests in test/telemetry/test_tracing_backend.py always skip with reason Telemetry not initialized, even when OTel is installed and Ollama is running.

Found whilst working on PR #742. Flagging for @ajbozart as the original author of the telemetry tests.

Root Cause

mellea/telemetry/tracing.py sets _tracer_provider at module import time based on the MELLEA_TRACE_BACKEND env var:

_tracer_provider = None
if _OTEL_AVAILABLE and (_TRACE_APPLICATION_ENABLED or _TRACE_BACKEND_ENABLED):
    _tracer_provider = _setup_tracer_provider()

The test's setup_telemetry fixture uses MonkeyPatch.setenv("MELLEA_TRACE_BACKEND", "true"), but by the time the fixture runs, tracing.py is already imported and _tracer_provider is already None. setenv cannot retroactively change a module-level constant.

The span_exporter fixture then checks if provider is None: pytest.skip(...) — which always fires.

Attempted Fix

Patching tracing._backend_tracer and tracing._TRACE_BACKEND_ENABLED directly in the fixture (bypassing the env var) gets 4/6 tests passing. The remaining 2 have deeper design issues:

  • test_span_duration_captures_async_operation: calls trace.get_tracer_provider().force_flush() on the global OTel provider, not the test provider — flushes the wrong thing. Span name search for "chat" may also not match.
  • test_context_propagation_parent_child: creates its parent span via trace.get_tracer(__name__) (global provider) but backend spans go to the test provider. Parent span never reaches InMemorySpanExporter. Context propagation assertion requires both spans on the same provider.

Suggested Fix

In span_exporter fixture, directly patch the module internals rather than relying on env var:

@pytest.fixture
def span_exporter():
    from mellea.telemetry import tracing

    provider = TracerProvider()
    exporter = InMemorySpanExporter()
    provider.add_span_processor(SimpleSpanProcessor(exporter))
    test_tracer = provider.get_tracer("mellea.backend")

    old_tracer = tracing._backend_tracer
    old_enabled = tracing._TRACE_BACKEND_ENABLED
    tracing._backend_tracer = test_tracer
    tracing._TRACE_BACKEND_ENABLED = True

    yield exporter

    exporter.clear()
    tracing._backend_tracer = old_tracer
    tracing._TRACE_BACKEND_ENABLED = old_enabled

test_context_propagation_parent_child additionally needs to create its parent span using the test tracer (not trace.get_tracer(__name__)) so both spans land on the same provider.

Verification

With the partial fix above, 4/6 tests pass against a live Ollama instance. The 2 remaining failures require the test bodies to be updated.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions