feat: add support for OTel context propagation and harmonized spans (v3.x) #1659

feywind · 2022-11-28T22:50:59Z

Work in progress, converting the existing OTel support to use context propagation and more finely grained spans.

weyert · 2022-12-08T13:49:37Z

@feywind Any way I can help with progressing this PR?

feywind · 2022-12-08T19:30:25Z

@weyert Hey, glad to see you're interested :D This is an effort to sync across all the supported languages, so it might be a while before it's fully merged/released. When it's a bit further along though, I'd be interested in testing help, since you're one of the people who's more into OTel usage. This is my main project right now, so I'm hoping I can get a preview branch going soon, at the least.

weyert · 2022-12-08T20:51:12Z

Awesome happy to try out a preview version when it is available :)

generated-files-bot · 2022-12-12T21:19:55Z

Warning: This pull request is touching the following templated files:

.kokoro/continuous/node12/samples-test.cfg - .kokoro files are templated and should be updated in synthtool
.kokoro/presubmit/node12/samples-test.cfg - .kokoro files are templated and should be updated in synthtool
.kokoro/samples-test.sh - .kokoro files are templated and should be updated in synthtool

…erly

…make one

hongalex

Looks good mostly, just some comments about ways to potentially help with readability

hongalex · 2023-05-05T19:44:53Z

samples/openTelemetryTracing.js

@@ -32,103 +32,107 @@

 const SUBSCRIBER_TIMEOUT = 10;

+// [START opentelemetry_tracing]


nit: we should standardize on this region tag at some point before submitting

Yeah, I think the person who originally made this just came up with one? I think maybe it deserves several samples, e.g. one for Cloud Trace, another for console output, etc.

I think we shouldn't do console output and just focus on Cloud Trace as the default sample (since that's also what our open telemetry team's documentation points to).

I do think that's the primary thing we want to help people with, 'cause it's probably what they want to do. But I do think we ought to at least e.g. have a Medium article talking about how to output to console, Grafana, etc. Those will be useful to people too.

src/telemetry-tracing.ts

src/subscriber.ts

src/telemetry-tracing.ts

src/subscriber.ts

test/telemetry-tracing.ts

src/telemetry-tracing.ts

feywind · 2023-08-15T18:48:08Z

I'm going to go ahead and base this off the 3.x branch to get it merged into that, and then I'll up-port it to 4.x.

hongalex · 2023-08-17T00:40:18Z

samples/openTelemetryTracing.js

@@ -32,103 +32,107 @@

 const SUBSCRIBER_TIMEOUT = 10;

+// [START opentelemetry_tracing]


I think we shouldn't do console output and just focus on Cloud Trace as the default sample (since that's also what our open telemetry team's documentation points to).

samples/openTelemetryTracing.js

SimenB · 2023-08-18T10:23:10Z

package.json

@@ -51,8 +51,8 @@
    "@google-cloud/precise-date": "^3.0.0",
    "@google-cloud/projectify": "^3.0.0",
    "@google-cloud/promisify": "^2.0.0",
-    "@opentelemetry/api": "^1.0.0",
-    "@opentelemetry/semantic-conventions": "~1.3.0",
+    "@opentelemetry/api": "~1.1.0",


any particular reason these are locked down from ^?

Yeah, it had to do with Node 12 compatibility. I found that there were some semver issues, in practice, though I'll be revisiting this for the 4.x version of the PR, so I'll probably recheck these for 3.x too.

approved code changes but still need to review tests

weyert · 2023-09-13T14:50:21Z

Can't wait for this one to get merged. I have run it locally and works well. I haven't been able to run it in the wild as we had some maddening memory issues but that might have been better

I assume it adheres to the messaging outline of opentelemetry? I have to admit I don't fully understand that one

feywind · 2023-09-13T20:08:06Z

Can't wait for this one to get merged.

You and I both :)

I have run it locally and works well. I haven't been able to run it in the wild as we had some maddening memory issues but that might have been better

Memory issue caused by the OTel PR or something unrelated?

I assume it adheres to the messaging outline of opentelemetry? I have to admit I don't fully understand that one

Such as it is, yeah. The inner workings of the GCP Pub/Sub libraries are represented in the span layout, but I don't think that's particularly specified by OTel. It's our intent to support the standard span attributes and W3C trace propagation attribute.

TBD if the span structure will be identical across client libraries, like C++ has a different thing going that's based on their method call structure.

Given that the 3.x release machinery is just about going, I'm planning to bring this PR up to the main branch at some point and then back-port the finished one to 3.x.

weyert · 2023-09-13T20:51:49Z

I have run it locally and works well. I haven't been able to run it in the wild as we had some maddening memory issues but that might have been better

Memory issue caused by the OTel PR or something unrelated?

Oh no, work code related issue. Nothing related to this PR sorry for confusing you.

I assume it adheres to the messaging outline of opentelemetry? I have to admit I don't fully understand that one

Such as it is, yeah. The inner workings of the GCP Pub/Sub libraries are represented in the span layout, but I don't think that's particularly specified by OTel. It's our intent to support the standard span attributes and W3C trace propagation attribute.

TBD if the span structure will be identical across client libraries, like C++ has a different thing going that's based on their method call structure.

Yeah, I understood their is this messaging semantic convention they are working on, like https://github.com/open-telemetry/semantic-conventions/blob/main/docs/messaging/messaging-spans.md

Makes sense the span structure is the same structure between libraries :))

Given that the 3.x release machinery is just about going, I'm planning to bring this PR up to the main branch at some point and then back-port the finished one to 3.x.

Makes sense but to be honest I can also fully understand it's a v4 thing

feywind · 2024-03-20T15:48:58Z

Will re-open this later when the 4.x one is done.

feywind added the do not merge Indicates a pull request not ready for merge, due to either quality or timing. label Nov 28, 2022

feywind requested review from a team as code owners November 28, 2022 22:51

product-auto-label bot added size: l Pull request size is large. api: pubsub Issues related to the googleapis/nodejs-pubsub API. size: xl Pull request size is extra large. and removed size: l Pull request size is large. labels Nov 28, 2022

feywind force-pushed the otel-2 branch from b430593 to 6e89f28 Compare December 12, 2022 21:19

product-auto-label bot added size: l Pull request size is large. and removed size: xl Pull request size is extra large. labels Dec 12, 2022

feywind added 8 commits December 12, 2022 16:28

feat: use the propagation inject framework for OpenTelemetry publishes

0d9c0b2

feat: add propagation extraction to subscriber for OpenTelemetry

4f3daa9

feat: work in progress on adding sub-spans

80d8b65

add more spans

cbd5992

wrap subscriber on() handlers to get better receive process spans

75acbab

update otel core version for tests

bb6cf85

fix: change some span names for doc updates

5cddaed

fix: move emitter wrapper out one level, to catch user functions prop…

d523b1f

…erly

feywind force-pushed the otel-2 branch from 6e89f28 to 7f4dc75 Compare December 12, 2022 21:31

product-auto-label bot added size: xl Pull request size is extra large. and removed size: l Pull request size is large. labels Dec 12, 2022

feywind force-pushed the otel-2 branch from 7f4dc75 to d523b1f Compare December 12, 2022 21:32

feywind added 4 commits December 14, 2022 13:34

chore: merge remote-tracking branch 'remotes/origin/main' into otel-2

fbe29a2

fix: don't send anything besides what's actually needed during publish

197a214

fix: close the publish span at RPC time, not up front

a36cfce

fix: delay caching of the trace provider so the user has a chance to …

5c4fef6

…make one

hongalex reviewed Aug 15, 2023

View reviewed changes

feywind changed the base branch from main to legacy-v3 August 15, 2023 18:46

Merge branch 'legacy-v3' into otel-2

68335e5

feywind added 7 commits August 15, 2023 16:38

fix: include batch size with each published message span

0f2dc41

chore: fix lint

1ba495a

fix: separate out the getter and setter classes

49b0bc3

chore: rename span related objects in subscriber

12f400d

chore: rename SpanMaker

febcd16

docs: improved comments and lint fixes

a471875

docs: more class/comment updating

a37194d

hongalex reviewed Aug 17, 2023

View reviewed changes

fix: missing spans caused by renames; also do more renames

9ec1fdf

SimenB reviewed Aug 18, 2023

View reviewed changes

feywind added 4 commits August 18, 2023 14:39

tests: more renaming issues

5f33931

tests: add otel test for FlowControlledPublisher

f167885

tests: add lease manager tests for otel

feeccf1

tests: add SubscriberSpans unit tests

b218d5f

hongalex previously approved these changes Sep 7, 2023

View reviewed changes

Merge branch 'legacy-v3' into otel-2

2471c16

feywind mentioned this pull request Sep 20, 2023

feat: add support for OTel context propagation and harmonized spans #1833

Open

feywind changed the title ~~feat: add proper OpenTelemetry support to Node Pub/Sub~~ feat: add support for OTel context propagation and harmonized spans Sep 20, 2023

feywind changed the title ~~feat: add support for OTel context propagation and harmonized spans~~ feat: add support for OTel context propagation and harmonized spans (v3.x) Sep 20, 2023

omBratteng mentioned this pull request Oct 17, 2023

feat: opentelemetry tracing in daily-api dailydotdev/daily-api#1482

Merged

feywind closed this Mar 20, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add support for OTel context propagation and harmonized spans (v3.x) #1659

feat: add support for OTel context propagation and harmonized spans (v3.x) #1659

feywind commented Nov 28, 2022 •

edited

weyert commented Dec 8, 2022

feywind commented Dec 8, 2022

weyert commented Dec 8, 2022

generated-files-bot bot commented Dec 12, 2022

hongalex left a comment

hongalex May 5, 2023

feywind Aug 15, 2023

hongalex Aug 17, 2023

feywind Aug 17, 2023

feywind commented Aug 15, 2023

hongalex Aug 17, 2023

SimenB Aug 18, 2023

feywind Aug 18, 2023

weyert commented Sep 13, 2023

feywind commented Sep 13, 2023

weyert commented Sep 13, 2023

feywind commented Mar 20, 2024

		@@ -32,103 +32,107 @@

		const SUBSCRIBER_TIMEOUT = 10;

		// [START opentelemetry_tracing]

feat: add support for OTel context propagation and harmonized spans (v3.x) #1659

feat: add support for OTel context propagation and harmonized spans (v3.x) #1659

Conversation

feywind commented Nov 28, 2022 • edited

weyert commented Dec 8, 2022

feywind commented Dec 8, 2022

weyert commented Dec 8, 2022

generated-files-bot bot commented Dec 12, 2022

hongalex left a comment

Choose a reason for hiding this comment

hongalex May 5, 2023

Choose a reason for hiding this comment

feywind Aug 15, 2023

Choose a reason for hiding this comment

hongalex Aug 17, 2023

Choose a reason for hiding this comment

feywind Aug 17, 2023

Choose a reason for hiding this comment

feywind commented Aug 15, 2023

hongalex Aug 17, 2023

Choose a reason for hiding this comment

SimenB Aug 18, 2023

Choose a reason for hiding this comment

feywind Aug 18, 2023

Choose a reason for hiding this comment

weyert commented Sep 13, 2023

feywind commented Sep 13, 2023

weyert commented Sep 13, 2023

feywind commented Mar 20, 2024

feywind commented Nov 28, 2022 •

edited