Pass OTEL_EXPORTER_OTLP_HEADERS env var to mcpg container for authenticated OTel export

[lpcox]

Summary

When observability.otlp is configured for a workflow, gh-aw should pass OTEL_EXPORTER_OTLP_HEADERS as an environment variable to the mcpg container. This is the standard OTel mechanism for providing authentication headers to OTLP exporters, and it avoids putting security-sensitive credentials in the stdin JSON config.

Additionally, the headers field should be removed from the gateway config spec and JSON schema for gateway.opentelemetry, since headers containing auth credentials should not flow through the stdin JSON config pipe.

Background

mcpg (MCP Gateway) already reads:

• ✅ OTEL_EXPORTER_OTLP_ENDPOINT — used as the default for --otlp-endpoint CLI flag
• ✅ OTEL_SERVICE_NAME — used as the default for --otlp-service-name CLI flag
• ❌ OTEL_EXPORTER_OTLP_HEADERS — not read; headers currently only come from stdin JSON config (gateway.opentelemetry.headers) or TOML config (gateway.tracing.headers)

This creates a gap: when gh-aw configures OTel export via observability.otlp, the endpoint reaches mcpg via env var, but the authentication headers do not. The export fails with 401 for any collector requiring auth (e.g., Sentry, Datadog, Grafana Cloud).

Proposed Changes

1. gh-aw side

• When observability.otlp is configured in a workflow frontmatter, pass the headers as the OTEL_EXPORTER_OTLP_HEADERS environment variable to the mcpg container, alongside OTEL_EXPORTER_OTLP_ENDPOINT which is presumably already passed.
• Remove headers from the gateway.opentelemetry config spec and JSON schema. Auth credentials should not be passed through the stdin JSON config. The standard OTEL_EXPORTER_OTLP_HEADERS env var is the correct mechanism — it's isolated inside the container and follows the OTel specification.

2. mcpg side (companion PR in gh-aw-mcpg)

Add support for reading OTEL_EXPORTER_OTLP_HEADERS as a fallback when gateway.opentelemetry.headers / gateway.tracing.headers is not set in the config file. This keeps the env var approach consistent with how OTEL_EXPORTER_OTLP_ENDPOINT already works.

Why env var instead of stdin config for headers?

OTEL_EXPORTER_OTLP_HEADERS typically carries authentication credentials (Bearer tokens, API keys). Using env vars inside a container is:

1. More secure than CLI args — not visible in ps output
2. Standard OTel convention — all OTel SDKs and collectors expect this env var
3. Isolated in containers — env is not visible to other processes outside the container
4. Simpler for gh-aw — no need to wire secrets through the stdin JSON config pipe

Removing headers from the gateway config spec/schema ensures there is one canonical way to pass auth headers (the env var), reducing the attack surface and simplifying the configuration model.

Format (per OTel spec)

Per the OTLP Exporter Configuration spec:

OTEL_EXPORTER_OTLP_HEADERS will contain a list of key value pairs, and these are expected to be represented in a format matching to the W3C Baggage, i.e., key1=value1,key2=value2. Semi-colon delimited metadata is not supported.

Example:

OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer sentry_abc123,X-Custom-Header=value"

References

• OTel SDK Environment Variables spec
• OTLP Exporter Configuration Options
• W3C Baggage header format
• mcpg tracing code: internal/cmd/tracing.go, internal/config/config_tracing.go
• mcpg gateway config spec: gateway.opentelemetry.headers field in stdin JSON schema