Skip to content

fix(cli-v3): forward TRIGGER_WORKER_INSTANCE_NAME into the managed run process env#4213

Closed
brentshulman-silkline wants to merge 2 commits into
triggerdotdev:mainfrom
brentshulman-silkline:fix/forward-worker-instance-name-to-run-process
Closed

fix(cli-v3): forward TRIGGER_WORKER_INSTANCE_NAME into the managed run process env#4213
brentshulman-silkline wants to merge 2 commits into
triggerdotdev:mainfrom
brentshulman-silkline:fix/forward-worker-instance-name-to-run-process

Conversation

@brentshulman-silkline

Copy link
Copy Markdown

Problem

On self-hosted Kubernetes, task-run spans and logs exported over OTLP to an
off-node collector (e.g. a central Datadog Agent) have an empty host. Datadog
tags every such span with issue_type:empty_hostname. There is no supported way
for a user's trigger.config.ts telemetry resource to recover the node, because
the node identity never reaches the run process.

Root cause

The Kubernetes workload manager already injects the node name into the
run-controller container via the downward API
(apps/supervisor/src/workloadManager/kubernetes.ts):

{ name: "TRIGGER_WORKER_INSTANCE_NAME",
  valueFrom: { fieldRef: { fieldPath: "spec.nodeName" } } }

But the actual task-run worker is fork()ed with an explicit, replaced env
built from RunnerEnv.gatherProcessEnv() (an allowlist) plus the run's env
(packages/cli-v3/src/entryPoints/managed/taskRunProcessProvider.ts
packages/cli-v3/src/executions/taskRunProcess.ts). gatherProcessEnv() returns
only NODE_ENV, NODE_EXTRA_CA_CERTS, OTEL_EXPORTER_OTLP_ENDPOINT, TRIGGER_OTEL_EXPORTER_OTLP_ENDPOINT, UV_USE_IO_URING, so
TRIGGER_WORKER_INSTANCE_NAME (and container-level vars generally) never reach
process.env in the run process. The node identity is present in the pod but
dropped before it can be used.

Fix

Add TRIGGER_WORKER_INSTANCE_NAME to the env returned by
RunnerEnv.gatherProcessEnv() (it is already in the Env schema and has a
getter). One line + comment; see change.diff.

With this, a user can map it in their trigger.config.ts:

const nodeName = process.env.TRIGGER_WORKER_INSTANCE_NAME; // spec.nodeName on k8s
telemetry: {
  resource: resourceFromAttributes(
    nodeName ? { "host.name": nodeName, "k8s.node.name": nodeName } : {}
  ),
  // ...
}

which clears empty_hostname and attributes the run to its actual node.

Scope / risk

  • Additive: forwards one already-present, non-secret identity var.
  • No behavior change when unset (the existing undefined filter drops it).
  • Docker workload manager already passes TRIGGER_WORKER_INSTANCE_NAME to the
    runner (workloadManager/docker.ts), so this aligns k8s runs with that.

Changeset

trigger.dev: patch — see changeset.md.

@changeset-bot

changeset-bot Bot commented Jul 9, 2026

Copy link
Copy Markdown

🦋 Changeset detected

Latest commit: dcd100b

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 27 packages
Name Type
trigger.dev Patch
@internal/dashboard-agent Patch
@trigger.dev/build Patch
@trigger.dev/core Patch
@trigger.dev/python Patch
@trigger.dev/react-hooks Patch
@trigger.dev/redis-worker Patch
@trigger.dev/rsc Patch
@trigger.dev/schema-to-json Patch
@trigger.dev/sdk Patch
@trigger.dev/database Patch
@trigger.dev/otlp-importer Patch
@trigger.dev/rbac Patch
@trigger.dev/sso Patch
@internal/cache Patch
@internal/clickhouse Patch
@internal/llm-model-catalog Patch
@internal/redis Patch
@internal/replication Patch
@internal/run-engine Patch
@internal/run-store Patch
@internal/schedule-engine Patch
@internal/testcontainers Patch
@internal/tracing Patch
@internal/tsql Patch
@internal/zod-worker Patch
@internal/sdk-compat-tests Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Hi @brentshulman-silkline, thanks for your interest in contributing!

This project requires that pull request authors are vouched, and you are not in the list of vouched users.

This PR will be closed automatically. See https://github.com/triggerdotdev/trigger.dev/blob/main/CONTRIBUTING.md for more details.

@github-actions github-actions Bot closed this Jul 9, 2026
@coderabbitai

coderabbitai Bot commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: cf4ec223-e437-4397-9bbc-2b9773eb4dfb

📥 Commits

Reviewing files that changed from the base of the PR and between 105f489 and dcd100b.

📒 Files selected for processing (2)
  • .changeset/forward-worker-instance-name.md
  • packages/cli-v3/src/entryPoints/managed/env.ts

Walkthrough

This change forwards the TRIGGER_WORKER_INSTANCE_NAME environment variable into the managed run process environment within RunnerEnv.gatherProcessEnv(), which previously omitted it. Comments were added explaining that this value, sourced from the Kubernetes downward API spec.nodeName, allows OpenTelemetry spans and logs to carry host/node resource attributes when exported to an off-node OTLP collector. A changeset was added documenting this as a patch release for trigger.dev.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 potential issue.

Open in Devin Review

Comment on lines +5 to +14
Forward `TRIGGER_WORKER_INSTANCE_NAME` into the managed run process env

The Kubernetes workload manager already injects `TRIGGER_WORKER_INSTANCE_NAME`
(= `spec.nodeName`) into the run-controller container via the downward API, but
the managed run worker is forked with an explicit env allowlist
(`RunnerEnv.gatherProcessEnv()`) that dropped it. As a result the run process
could not attach the node as an OpenTelemetry `host.name` resource attribute, so
spans/logs exported to an off-node OTLP collector had no host (Datadog tags these
`issue_type:empty_hostname`). This forwards the value so `trigger.config.ts`
telemetry (or the worker itself) can surface the node/host.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changeset description uses internal jargon instead of user-facing language

The changeset body names internal infrastructure and code paths (RunnerEnv.gatherProcessEnv(), Kubernetes downward API, Datadog, OTLP collector) (.changeset/forward-worker-instance-name.md:5-14), violating the repo rule that changeset descriptions must be written for users, not maintainers.

Impact: Internal implementation details will appear verbatim in public release notes, confusing end users.

Repo rule violation details

AGENTS.md states: "Write the description for users, not maintainers. Both changesets and .server-changes/ notes ship verbatim in user-visible release notes. Lead with what changed for the user - one plain sentence describing behavior, not implementation, and never naming internal tools or infra."

The .server-changes/README.md:47-48 reinforces: "Never name internal tools or infra. Observability stacks, internal services, infra components, monitoring backends, CI surfaces, AWS specifics - none of these belong in user-facing notes."

The current changeset body mentions: Kubernetes workload manager, downward API, RunnerEnv.gatherProcessEnv(), OpenTelemetry, OTLP collector, Datadog, trigger.config.ts telemetry internals. All of these are internal implementation details.

Suggested change
Forward `TRIGGER_WORKER_INSTANCE_NAME` into the managed run process env
The Kubernetes workload manager already injects `TRIGGER_WORKER_INSTANCE_NAME`
(= `spec.nodeName`) into the run-controller container via the downward API, but
the managed run worker is forked with an explicit env allowlist
(`RunnerEnv.gatherProcessEnv()`) that dropped it. As a result the run process
could not attach the node as an OpenTelemetry `host.name` resource attribute, so
spans/logs exported to an off-node OTLP collector had no host (Datadog tags these
`issue_type:empty_hostname`). This forwards the value so `trigger.config.ts`
telemetry (or the worker itself) can surface the node/host.
Forward the worker/node name into managed run processes so custom telemetry exporters can identify which host executed each run
Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant