Unable to filter out noise from trace open telemetry

[TerryDieckmann]

I am currently investigating how we can best use OTel and Phobos in our software solution. During the implementation I can't really figure out how to filter out the messages below without adding an endless list of separate message filters.

The trace-actor-lifecycle option is disabled, but there are still plenty of internal akka (unnecessary) messages.

Using .NET 6, Akka.NET Cluster, Open Telemetry and Phobos.

[Aaronontheweb]

Thanks Terry - we'll take a look at this.

[TerryDieckmann]

Thanks Aaron,
It might be good to know that I use Phobos.Actor.Cluster 2.0.0-beta3. I've tried filtering out the bulk of (to me) unnecessary messages, but when I do that, half of the trace is no longer included because it is a child of akka.msg.recv GetShardHome.

[Aaronontheweb]

We have another issue related to trace filtering on Phobos 1.4 with #46 - so let's just say that this has been clearly identified as a "user experience" problem across all use cases ;)

We learned some lessons on how to handle filtering more robustly when we implemented type filters for security purposes inside Hyperion akkadotnet/akka.net#5329 (comment)

I'll work on improving the upfront filtering and also add an example for using an OTel Processor on the back-end.

[Aaronontheweb]

Wrote up our plans for addressing this on a related issue here: #46 (comment) - would love your feedback on it @TerryDieckmann

[Aaronontheweb]

Hi @TerryDieckmann - we have pushed new Phobos 2.0 binaries that have an API change that should nip this problem in the bud.

Please see the release notes here and follow the configuration instructions - I think this should help quite a bit: https://phobos.petabridge.com/articles/releases/RELEASE_NOTES.html#200-beta4httpssdkbincompublisherpetabridgeproductphobospackagesphobosactorversions200-beta4-and-150-beta1httpssdkbincompublisherpetabridgeproductphobospackagesphobosactorversions150-beta1-march-17th-2022

[TerryDieckmann]

I will have a look, thanks!

[TerryDieckmann]

I've implemented my own tracer filter and it works great! Traces are complete and not full of messages we don't really need. Thank you, Aaron!

[Aaronontheweb]

Glad to hear it!