Any docs on how to play nice with an OTEL tracing/metrics setup?

[pvogel1967]

I'd stayed away from using AsyncLocalStorage due to performance concerns for a while, but since Otel uses it anyway, using it for CLS shouldn't add any additional overhead. So...

I did some work late last week to get a request-scoped logger dependency out of all our services, nice boost in performance that! In local testing (where tracing and metrics from otel are turned off) things worked great. data stored in CLS at the start of a request is found in all logs.

However, when testing in our staging environment, where OTEL tracing and metrics are turned on the data stored in CLS is not found in logs, but data injected by the OTEL bunyan auto-instrumentation (i.e. traceId, etc.) was in the logs.

Ideally, I'd like both. I'd like to augment the data in CLS with the traceId for the incoming request.

Any guidance for how to make that happen correctly?

[Papooch]

Hi, as far as I'm concerned, I don't see why it would be an issue. Both OTEL and nestjs-cls use their own isolated instance of AsyncLocalStorage.

I have never had issues in applications where both nestjs-cls and OTEL was used. It might help if you shared how both of them are set up.

However, I have never used Bunyan as a logger, and considering how old it is (last update 3 years ago), there might be something in the OTEL/Bunyan instrumentation that breaks AsyncLocalStorage context propagation. A similar issue was present in some older express middlewares.

I can confidently say that the combination of OTEL and nestjs-cls works with the Winston and Pino loggers, so if it's not too much work, you could try swapping Bunyan for one of them and see if it helps.

---

By the way, if the only thing you use nestjs-cls for is to propagate the traceId to your logs, I would probably advise you just to use the OTEL context alone.

[pvogel1967]

Yeah, definitely adding additional data beyond traceId, was just wanting to get the trace and span into the logs as well. What I'm seeing is that the bunyan instrumentation is getting the traceId, etc. added to the logs, but that the context I'm trying to add with nestjs-cls is not appearing in the logs.

This is quite a large codebase, so swapping bunyan out for Winston or Pino is not trivial, but I may take a look at what it would take to do that...

Let me try to get to a reproducible scenario on my local box and we can go from there...

[Papooch]

Another thing that came to my mind is to check your Node version. AsyncLocalStorage was made stable in v16, but there were still some issues with propagation in certain situations and they fixed that since, so you might want to check with the latest release.

And the other thing is whether you need the Buyan auto-instrumentation by OTEL at all or if you could enrich the logs with the request context yourself.

If you can produce a mininal repro, I'll look into it on my end.

[Papooch]

@pvogel1967 Hi, have you eventually figured this out, or do you need some assistance?