Tracer.close() is not fully synchronous

[yurishkuro]

It was meant to be fully synchronous to guarantee that all spans are flushed, however it only blocks until the spans are handed off to the sender, which itself is async, so if close() is used from a command line tool or a hello-world app that exist immediately, the spans may not have time to flush to UDP port.

[kgrvamsi]

@yurishkuro do we need to append this time.sleep(2) when we write a production level application ?
Consider the scenario of distributed system where Agent/Collector/Query runs on Centralized host and we have code running on containers(roughly 30+) will the trace close needed based on the network lag for all these containers communicating to the centralized server?

[yurishkuro]

If your services support graceful shutdown they yes, you might need to add this sleep after calling tracer.close(). But most long-running server applications are simply killed when they are upgraded or relocated, so in practice this rarely matters since close() won't be called anyway. You might lose some spans not flushed yet.

[spsotirop]

Hello,
I am hijacking this thread since I am facing an issue with the reporting of the spans. I would like to integrate the library in a StackStorm project (the underlying stack includes the usage of gunicorn and eventlet). My approach is the instantiation of a jaeger client (as a member of a custom singleton class) and the reporting of the spans is done inside a decorator used in the functions I would like to measure. The problem is that I must use time.sleep(2) in order for the spans to reach the collector. Is there a different approach to overcome this?

def tracing_decorator(func):
    @wraps(func)
    def wrapper_func(self, *args, **kwargs):
        name = func.__name__
        class_name = self.__class__.__name__
        with Tracer.get_instance().tracer_client.start_span(name) as span:
            span.log_kv({'function': name, 'class': class_name})
            s = func(self, *args, **kwargs)
        time.sleep(2)
        return s
    return wrapper_func

[yurishkuro]

There is no different approach, I think, we just need to fix this specific issue.

[spsotirop]

Ok no problem, thanks for the prompt response

[obataku]

@yurishkuro is there a reason await-ing the Future from Tracer.close() (in the context of an event loop of course) would be insufficient to flush everything? waiting for the queue to drain (as Reporter._flush does with yield self.queue.join()) sounds like it would work but your comment

it only blocks until the spans are handed off to the sender

... as well as @nziebart's here

we could wait until the length of the queue is zero, but that does not necessarily mean that the background thread has finished sending all the data

... make me suspect I'm missing something

[yurishkuro]

@obataku I don't know the underlying cause, it doesn't seem to work.

[AudriusButkevicius]

Is this still relevant given the merged PR?

[kasium]

@obataku so if I understand your PR right, simply doing yield trace.stop() should help?