Tracing with async/CLI setups

[SonarSoftware]

I'm currently using Swoole, but I'm guessing it's a similar issue for ReactPHP. Is there any direction on getting it working on an environment like this today? Or any plans to implement?

[labbati]

Hi @SonarSoftware, we definitely have plans for ReactPHP as it came up a few times now, and I guess a similar solution might apply to Swoole as well, as you are suggesting.

I cannot provide an ETA at the moment, though. Given that I am not expert about RactPHP/Swoole internals (I only can assume that there is some separate thread/event loop under the hood) this would make it difficult for you to support tracing without some work on our side due to the following reasons that comes to my mind (@SammyK and @morrisonlevi feel free to jump in as you might have more experience than me with those libraries):

• at the moment we flush traces on a register_shutdown_function. This would not work in your case as your script (I assume) runs as long as the app runs.
• Even if you manual invoke GlobalTracer::get()->flush() in some after-response hook that is provided by Swoole/ReactPHP, the Tracer remains in an unpredictable state as it approaches serving next request.
• On top of that, and this might be the really tricky part, We use at the moment one global tracer instance (as per OpenTracing approach) and span coming from different requests I am pretty sure would end up mixed in a giant basket.

Again, I have never used such libraries so I can only talk from the top of my head.

[SonarSoftware]

Yes, it is tricky. There are some ways around the global tracer issue - Swoole for example has a sandbox that is used to separate per-request items out. That being said, it's still not easy and it would definitely take some work.

It might be worth taking a look at https://github.com/upscalesoftware/swoole-newrelic for some inspiration - I've been poking at it a bit seeing if we can do something similar with ddtrace.

[nesl247]

Any update on this? I ask because we are in the process of migrating from NewRelic and ran into this. It is not an issue we expected to show up.

[morrisonlevi]

I am not aware of any updates specific to ReactPHP or Swoole, but I know some of the underlying engineering challenges that assume a single global trace are being worked on.

One thing to set expectations on is what in these products gets traced. To that end you could help by sharing what features you are using. As an example, I found a React\Http\Server class -- I assume that users would want each new request that the server processes to get traced independently. Telling us which pieces of Swoole and ReactPHP are being used can help us know where to look in docs, examples, etc when this work gets prioritized.

Another thing worth knowing is what PHP version and which version of Swoole or ReactPHP you are using, to make sure our work is actually relevant to what our users need.

[nesl247]

I believe we're using Swoole v4.4.15 at this time with PHP 7.4. And yes, we would expect each new request that swoole processes to be traced independently.

Also, thanks for the fast update.

[SonarSoftware]

Yep, exactly. Just treat each request as independent. I think this might be fairly easy to do by hooking into the onRequest method.. I'm planning to take a look at this myself soon.

[mrtus]

HI @morrisonlevi @labbati Could you provide us with an update on the work being done to make this possible?

And would it maybe be possible to elaborate a little bit more on this:

Even if you manual invoke GlobalTracer::get()->flush() in some after-response hook ..., the Tracer remains in an unpredictable state as it approaches serving next request.

As I'm currently looking into getting our distributed services through queueing connected to each other with tracing (we already have our logs connected through the same way with a trace_id property). I would, just like it is already done with curl, attach the tracing data on the message when published and get that data from the message when consumed by a long running process worker. After this message is consumed, I would of course want to call a flush, but I'm reading here that the tracer would be in an unpredictable state. Can you elaborate what you mean with unpredictable, what are the side-effects of this, and would there be some sort of remenand traces that would tie multiple messages together?

[andi-xd]

Hi, any updates about this feature?

Thanks!

[labbati]

Hi @Anderson-Flores! At the moment we were not able to schedule the work for this and we cannot commit to a specific timeline. We will update as soon as possible.

[wregis]

I guess I've got a similar problem with Laravel Horizon running for days on a server. It would be nice to trace jobs individually.

[alexjose]

Swoole and ReactPHP is gaining more and more traction. At the same time, we have Laravel Octane working with Swoole under the hood. It would be very great to speed up the support of the coroutines in Datadog.