Skip to content

bug: acouchbase query span is started but never ended when the awaiting task is cancelled #67

@alex-raw

Description

@alex-raw

Summary

With an OTel tracer configured on ClusterOptions, cancelling the task
that awaits a cluster.query(...).rows() iterator leaves the query
span started but never ended. on_start fires; on_end does not, so
the span is never exported.

Skimming acouchbase/n1ql.py, the __anext__ exception clauses cover
StopAsyncIteration, QueueEmpty, CouchbaseException, and Exception
but not asyncio.CancelledError (<: BaseException).

Reproduction

import asyncio
from typing import Optional

from opentelemetry import trace
from opentelemetry.context import Context
from opentelemetry.sdk.trace import ReadableSpan, Span, SpanProcessor, TracerProvider

from acouchbase.cluster import AsyncCluster
from couchbase.auth import PasswordAuthenticator
from couchbase.observability.otel_tracing import get_otel_tracer
from couchbase.options import ClusterOptions


class LifecycleProbe(SpanProcessor):
    def __init__(self) -> None:
        self.started: list[str] = []
        self.ended: list[str] = []

    def on_start(self, span: Span, parent_context: Optional[Context] = None) -> None:
        self.started.append(span.name)

    def on_end(self, span: ReadableSpan) -> None:
        self.ended.append(span.name)

    def shutdown(self) -> None: ...
    def force_flush(self, timeout_millis: int = 30000) -> bool: return True


async def main() -> None:
    probe = LifecycleProbe()
    provider = TracerProvider()
    provider.add_span_processor(probe)
    trace.set_tracer_provider(provider)

    cluster = await AsyncCluster.connect(
        "couchbase://<host>",
        ClusterOptions(
            PasswordAuthenticator("<user>", "<password>"),
            tracer=get_otel_tracer(),
        ),
    )

    # First row arrives well after the cancellation below.
    slow_query = "SELECT COUNT(*) FROM ARRAY_RANGE(0, 50000000) AS r"

    async def run() -> None:
        async for _ in cluster.query(slow_query).rows():
            pass

    task = asyncio.create_task(run())
    await asyncio.sleep(0.2)
    task.cancel()
    try:
        await task
    except asyncio.CancelledError:
        pass

    print(f"query started: {probe.started.count('query')}")
    print(f"query ended:   {probe.ended.count('query')}")


asyncio.run(main())

Expected

query started: 1, query ended: 1.

Observed

query started: 1, query ended: 0.

Environment

  • couchbase 4.6.1
  • opentelemetry-api / opentelemetry-sdk 1.42.1
  • Python 3.14.2, Couchbase Server 8.0.0-3777-enterprise, Linux x86_64

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions