Feat: Opentelemetry for TS SDK

[jishnundth]

Description

1. OTel Instrumentor for Typescript SDK (Through monkey patching)
2. OTel example for TS SDK instrumentor
3. OTel environment variables for TS SDK

Type of change

• New feature (non-breaking change which adds functionality)

[vercel]

@jishnundth is attempting to deploy a commit to the Hatchet Team on Vercel.

A member of the Team first needs to authorize it.

[jishnundth]

Hey @mrkaye97,

Please take a look at this, when you have time.

The current approach follows monkey patching files for instrumentation during load time, which is the standard pattern used in OTel TypeScript examples as well.

(References)

1. https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/packages/instrumentation-amqplib/src/amqplib.ts
2. https://github.com/open-telemetry/opentelemetry-js-contrib/blob/main/packages/instrumentation-graphql/src/instrumentation.ts

However, the issue with this approach is that register instrumentation must occur first, before loading any other libraries. This is because Node.js caches loaded libraries. If a library that needs to be instrumented is imported before the instrumentation is registered, the patching will not occur. The rest of the program will then use the cached version of the library, which does not contain the patched OTel code. As a result, the spans internally added in the patched module will not appear in HyperDX.

You might already know this, just want to give a heads up on how opentelemetry works for JS.

NOTE: Also, for testing with the examples, the code has to built, packed and installed in another node_modules directory. Because patching follows the path of the file inside a node_module directory.

I also had to edit handleStartStepRun to throw the errors upstream, to capture the stacktrace in OTel.

[jishnundth]

It's working pretty good, same as python OTel

[mrkaye97]

Great, thanks! Will take a look soon :)

[mrkaye97]

left a couple small comments but basically looks good to me 🥳

will let @grutt take a look through since he has some more context here :) great work!

[mrkaye97]

btw just confirming - you've tested propagating the span context 1) from trigger -> run (so the trigger workflow run and the run workflow are part of the same trace, with the run nested as a span under the trigger) and 2) from parent -> child (when a task spawns a child the child should be nested under the parent)?

[jishnundth]

btw just confirming - you've tested propagating the span context 1) from trigger -> run (so the trigger workflow run and the run workflow are part of the same trace, with the run nested as a span under the trigger) and 2) from parent -> child (when a task spawns a child the child should be nested under the parent)?

Thanks for the message.

Case 1 is properly tested. -> run is propagating correctly and showing proper nesting in the trace.
I missed out on Case 2. Will test and confirm shortly

[jishnundth]

@mrkaye97 Both cases work seemlessly. Please check the screenshots below

Run Workflow

This is the example in the PR. Where the workflow contains 4 tasks. 2 autoinstrumented ones (no separate spans), and 2 tasks having their own spans in the task.

Parent Child Workflow

A parent task spawing 3 child tasks. parent-orchestartion and spawn-child-int are spans inside the parent task.
child-compute is a span inside the child task.

Let me know, if there's issues.

Also when you're testing please remember the note I shared above: #2828 (comment)

[mrkaye97]

@mrkaye97 Both cases work seemlessly. Please check the screenshots below

Run Workflow

This is the example in the PR. Where the workflow contains 4 tasks. 2 autoinstrumented ones (no separate spans), and 2 tasks having their own spans in the task.

## Parent Child Workflow A parent task spawing 3 child tasks. _parent-orchestartion_ and _spawn-child-int_ are spans inside the parent task. _child-compute_ is a span inside the child task. Let me know, if there's issues.

Also when you're testing please remember the note I shared above: #2828 (comment)

Amazing!

[mnafees]

Hey @jishnundth, sorry this is taking so long. We just merged some major changes in main so could you please rebase with the latest changes and I will make sure this gets shipped this week itself. Thanks so much.

[jishnundth]

Hey @jishnundth, sorry this is taking so long. We just merged some major changes in main so could you please rebase with the latest changes and I will make sure this gets shipped this week itself. Thanks so much.

No worries. I'm working on fixing things with the latest changes. Will keep you posted

[jishnundth]

@mnafees please check now

[grutt]

super close! a few questions and then we'll need to resolve conficts/lint

[jishnundth]

PTAL, the requested changes are added

[grutt]

one last comment to add comments on core sdk functions to prevent future breakage

[jishnundth]

one last comment to add comments on core sdk functions to prevent future breakage

Done!

[promptless-for-oss]

📝 Documentation updates detected!

Updated existing suggestion: Document OpenTelemetry support for Python, TypeScript, and Go SDKs

---

Tip: Configure how Promptless handles changelogs in Agent Settings 📋