feat: user-defined OTLP attributes with template expansion per workflow phase#33846
Conversation
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
|
@copilot update prefix gh-aw in expressions |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Done — all |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Done — renamed the |
|
@copilot remove the templating support, user can use the GitHub actions expressions |
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Done — removed the |
|
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Done in two commits:
|
|
✅ PR Code Quality Reviewer completed the code quality review. |
|
🧠 Matt Pocock Skills Reviewer has completed the skills-based review. ✅ |
|
❌ Design Decision Gate 🏗️ failed during design decision gate check. |
|
🧪 Test Quality Sentinel completed test quality analysis. |
|
@copilot revert the change gh-aw -> GitHub.aw |
There was a problem hiding this comment.
Pull request overview
Adds workflow-configurable custom OTLP span attributes via frontmatter, propagated to the runtime through a new GH_AW_OTLP_ATTRIBUTES env var and masked in runner logs, with additional updates to OTLP naming/service metadata.
Changes:
- Extend
observability.otlpfrontmatter schema to supportattributes: { <key>: <template> }and encode it intoGH_AW_OTLP_ATTRIBUTESfor the runtime. - Add masking support for custom attribute values via a new
mask_otlp_attributes.shand compiler-injected masking steps. - Improve observability config merging for imports (endpoints + attributes), and update OTEL service/span naming from
gh-aw.*togithub.aw.*in several places (tests/workflows/runtime).
Show a summary per file
| File | Description |
|---|---|
| pkg/workflow/frontmatter_types.go | Adds OTLPConfig.Attributes to the frontmatter schema and documents template usage. |
| pkg/workflow/observability_otlp.go | Collects/merges/encodes custom OTLP attributes into GH_AW_OTLP_ATTRIBUTES; updates default OTEL service name. |
| pkg/workflow/compiler_orchestrator_workflow.go | Merges imported OTLP endpoints and custom attributes into RawFrontmatter with correct precedence. |
| pkg/parser/import_field_extractor.go | Extends import-side observability merge to include custom attributes (first import wins). |
| pkg/workflow/compiler_yaml_main_job.go | Injects a masking step for GH_AW_OTLP_ATTRIBUTES values when present. |
| pkg/workflow/compiler_safe_outputs_job.go | Injects a masking step for GH_AW_OTLP_ATTRIBUTES values when present. |
| pkg/workflow/compiler_activation_job_builder.go | Injects a masking step for GH_AW_OTLP_ATTRIBUTES values when present. |
| actions/setup/sh/mask_otlp_attributes.sh | New script to ::add-mask:: each attribute value from GH_AW_OTLP_ATTRIBUTES. |
| actions/setup/js/send_otlp_span.cjs | Runtime support for reading/appending custom attributes to emitted spans. |
| actions/setup/js/emit_outcome_spans.cjs | Updates outcome span naming/attribute keys; appends custom OTLP attributes to the summary span. |
| actions/setup/js/action_setup_otlp.cjs | Updates setup span naming documentation to github.aw.*. |
| actions/setup/js/action_conclusion_otlp.cjs | Updates conclusion span naming to github.aw.*. |
| pkg/workflow/observability_otlp_test.go | Updates tests for new service name and adds tests for custom attribute extraction/injection. |
| pkg/workflow/observability_otlp_mask_script_test.go | Adds tests for the new masking script behavior. |
| actions/setup/js/emit_outcome_spans.test.cjs | Updates tests for renamed outcome span names/attribute keys. |
| actions/setup/js/action_otlp.test.cjs | Updates tests for renamed setup/conclusion span names and attribute keys. |
| actions/setup/js/action_conclusion_otlp.test.cjs | Updates tests for renamed conclusion span naming behavior. |
| .github/workflows/*.lock.yml | Bulk updates OTEL service names in locked workflows to the new github.aw.* convention. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comments suppressed due to low confidence (1)
actions/setup/js/send_otlp_span.cjs:1052
- This docstring says OTEL_SERVICE_NAME defaults to "gh-aw", but the compiler now defaults/injects "github.aw". Please update the default mentioned here (and/or the fallback value used in code) so the runtime documentation matches the compiler behavior.
- Files reviewed: 239/240 changed files
- Comments generated: 8
| func TestCollectOTLPCustomAttributes(t *testing.T) { | ||
| tests := []struct { | ||
| name string | ||
| frontmatter map[string]any | ||
| want map[string]string | ||
| }{ |
| func TestMaskOTLPAttributesScript(t *testing.T) { | ||
| if runtime.GOOS != "linux" { | ||
| t.Skip("requires Linux bash script behavior") | ||
| } | ||
|
|
||
| _, file, _, ok := runtime.Caller(0) | ||
| require.True(t, ok, "runtime.Caller should resolve the current test file") | ||
|
|
||
| scriptPath := filepath.Join(filepath.Dir(file), "..", "..", "actions", "setup", "sh", "mask_otlp_attributes.sh") |
| // extractOTLPAttributesFromObsMap reads the custom OTLP attributes map from a | ||
| // raw observability section (as parsed from an import's frontmatter). Only | ||
| // string values are accepted; non-string values are silently ignored. | ||
| // Returns nil when the field is absent or empty. | ||
| // | ||
| // Note: this intentionally duplicates the logic of | ||
| // workflow.extractOTLPCustomAttributesFromObsMap. The parser package must not | ||
| // import the workflow package (circular-dependency risk), so the helper lives | ||
| // here as a local copy. Both implementations must stay in sync. | ||
| func extractOTLPAttributesFromObsMap(obs map[string]any) map[string]string { | ||
| if obs == nil { | ||
| return nil | ||
| } | ||
| otlpAny, ok := obs["otlp"] | ||
| if !ok { | ||
| return nil | ||
| } | ||
| otlpMap, ok := otlpAny.(map[string]any) | ||
| if !ok { | ||
| return nil | ||
| } | ||
| attrsAny, ok := otlpMap["attributes"] | ||
| if !ok { | ||
| return nil | ||
| } | ||
| attrsMap, ok := attrsAny.(map[string]any) | ||
| if !ok { | ||
| return nil | ||
| } | ||
| result := make(map[string]string, len(attrsMap)) | ||
| for k, v := range attrsMap { | ||
| if s, ok := v.(string); ok && k != "" { | ||
| result[k] = s | ||
| } | ||
| } | ||
| return result | ||
| } |
| /** | ||
| * Build additional OTLP attribute objects from the GH_AW_OTLP_ATTRIBUTES | ||
| * environment variable. | ||
| * | ||
| * Attribute values are used as-is (use GitHub Actions expressions like | ||
| * `${{ vars.MY_VALUE }}` in workflow frontmatter for dynamic values). | ||
| * Attributes whose value is an empty string are omitted. When no custom | ||
| * attributes are configured, an empty array is returned. | ||
| * | ||
| * @returns {Array<{key: string, value: object}>} | ||
| */ | ||
| function buildCustomOTLPAttributes() { | ||
| const customDefs = parseOTLPCustomAttributes(); | ||
| if (!customDefs) return []; | ||
|
|
||
| const result = []; | ||
| for (const [key, value] of Object.entries(customDefs)) { | ||
| if (typeof key !== "string" || !key || typeof value !== "string") continue; | ||
| if (value !== "") { | ||
| result.push(buildAttr(key, value)); | ||
| } | ||
| } | ||
| return result; |
| const experimentAssignments = readExperimentAssignments(); | ||
| attributes.push(...buildExperimentAttributes(experimentAssignments)); | ||
| attributes.push(...buildEpisodeAttributesFromContext(awInfo, runId, runAttempt)); | ||
| // Append user-defined custom attributes from observability.otlp.attributes. | ||
| attributes.push(...buildCustomOTLPAttributes()); | ||
|
|
| // Append user-defined custom attributes from observability.otlp.attributes. | ||
| summaryAttributes.push(...buildCustomOTLPAttributes()); | ||
|
|
| // Attributes defines additional custom key-value string attributes to attach | ||
| // to every OTLP span emitted by this workflow (setup, agent, and conclusion). | ||
| // Values support template variables using {{ variable }} syntax, where the | ||
| // variable name is any OTLP attribute key already computed for the span | ||
| // (e.g. {{ github.aw.episode.id }}, {{ github.actor }}). | ||
| // | ||
| // Example – emit Langfuse session/user attributes alongside the standard ones: | ||
| // attributes: | ||
| // langfuse.session.id: "{{ github.aw.episode.id }}" | ||
| // session.id: "{{ github.aw.episode.id }}" | ||
| // langfuse.user.id: "{{ github.actor }}" | ||
| // user.id: "{{ github.actor }}" |
| func otelServiceName(workflowData *WorkflowData) string { | ||
| const defaultServiceName = "gh-aw" | ||
| const defaultServiceName = "github.aw" |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Template expansion was removed but documentation still claims it exists — this will break user workflows.
### Review Summary
Critical Issues (must fix before merge)
-
Misleading documentation — PR description and code comments describe
{{ variable }}template expansion, but commit ff24b74 removed this feature. Users will configure templates that don't work. -
Formatting failures — Two test files are not gofmt-formatted and will fail CI checks.
-
Breaking change — Default service name changed from
"gh-aw"to"github.aw"without migration guidance, breaking existing observability dashboards.
What Works Well
✅ Clean merge strategy for custom attributes from imports (main workflow wins, first import wins)
✅ Proper masking of attribute values via secret redaction API
✅ Comprehensive test coverage for the attribute collection and merging logic
Warning
Firewall blocked 1 domain
The following domain was blocked by the firewall during workflow execution:
proxy.golang.org
To allow these domains, add them to the
network.allowedlist in your workflow frontmatter:
network:
allowed:
- defaults
- "proxy.golang.org"See Network Configuration for more information.
🔎 Code quality review by PR Code Quality Reviewer · ● 1.2M
| // langfuse.session.id: "{{ github.aw.episode.id }}" | ||
| // session.id: "{{ github.aw.episode.id }}" | ||
| // langfuse.user.id: "{{ github.actor }}" | ||
| // user.id: "{{ github.actor }}" |
There was a problem hiding this comment.
Documentation claims template support that was removed: The comment describes {{ variable }} template expansion (lines 241-243), but template expansion was removed in commit ff24b74. Users will configure templates expecting them to work, but values will be used literally.
💡 Suggested fix
Update the comment to remove template references:
// Attributes is an optional map of custom OTLP attribute key-value pairs to attach
// to every OTLP span emitted by this workflow (setup, agent, and conclusion).
// Values are used as-is. Use GitHub Actions expressions like ${{ vars.MY_VALUE }}
// in workflow frontmatter for dynamic values.
//
// Example – emit Langfuse session/user attributes:
// attributes:
// langfuse.session.id: "${{ github.run_id }}"
// langfuse.user.id: "${{ github.actor }}"The current documentation will cause user confusion and misconfigured workflows.
| name string | ||
| frontmatter map[string]any | ||
| want map[string]string | ||
| }{ |
There was a problem hiding this comment.
Test code is not gofmt-formatted: This test block (lines 1598-1620+) is missing proper indentation, which will fail CI formatting checks.
💡 Suggested fix
Run gofmt -w pkg/workflow/observability_otlp_test.go to auto-format the file. The issue affects the entire TestCollectOTLPCustomAttributes function and its test table.
| _, file, _, ok := runtime.Caller(0) | ||
| require.True(t, ok, "runtime.Caller should resolve the current test file") | ||
|
|
||
| scriptPath := filepath.Join(filepath.Dir(file), "..", "..", "actions", "setup", "sh", "mask_otlp_attributes.sh") |
There was a problem hiding this comment.
Test code is not gofmt-formatted: This test block is missing proper indentation throughout, which will fail CI formatting checks.
💡 Suggested fix
Run gofmt -w pkg/workflow/observability_otlp_mask_script_test.go to auto-format the file.
|
|
||
| func otelServiceName(workflowData *WorkflowData) string { | ||
| const defaultServiceName = "gh-aw" | ||
| const defaultServiceName = "github.aw" |
There was a problem hiding this comment.
Breaking change without migration path: Changing the default OTEL service name from "gh-aw" to "github.aw" will break existing dashboards, alerts, and queries that filter by service.name.
💡 Suggested fix
Options:
- Keep the old default (
"gh-aw") and document the new naming convention as recommended but opt-in - Add this to release notes as a breaking change with migration guidance for users to update their dashboards/queries
- Support both formats temporarily with a deprecation period
This affects 240 workflow files in this PR, so the impact is significant.
There was a problem hiding this comment.
Skills-Based Review 🧠
Applied /tdd and /diagnose — requesting changes on incomplete feature implementation.
📋 Critical Issue: Template Expansion Not Implemented
Feature Advertised vs. Implemented
The PR description and documentation promise template variable expansion:
Documented behavior:
observability:
otlp:
attributes:
langfuse.session.id: "{{ gh-aw.episode.id }}" # Should expand at runtime
user.id: "{{ gh-aw.run.actor }}"Actual behavior: The implementation only passes static values through as-is. Template expansion is completely missing:
- No
expandOTLPAttributeTemplatefunction exists (claimed in PR description) buildCustomOTLPAttributes()has no parameter for existing span attributes- Tests only validate static attribute passthrough, not template expansion
Impact
Users who configure templates will get literal strings in their OTLP spans:
{"langfuse.session.id": "{{ gh-aw.episode.id }}"}instead of the resolved value:
{"langfuse.session.id": "ep-12345"}This breaks the entire use case for Langfuse session tracking.
What's Needed Before Merge
- Implement template expansion in
send_otlp_span.cjs - Add tests for template variable resolution
- Update function signature to accept existing attributes for template resolution
- Validate end-to-end that templates expand correctly in real workflow runs
✅ Positive Highlights
- ✅ Clean Go compiler integration with
GH_AW_OTLP_ATTRIBUTESenv var - ✅ Proper JSON encoding/decoding for attribute transport
- ✅ Good test structure for static attribute handling
- ✅ Documentation is clear about the intended behavior
Recommendation: Do not merge until template expansion is fully implemented and tested. The current PR delivers only half the feature.
🧠 Reviewed using Matt Pocock's skills by Matt Pocock Skills Reviewer · ● 1.9M
| // variable name is any OTLP attribute key already computed for the span | ||
| // (e.g. {{ github.aw.episode.id }}, {{ github.actor }}). | ||
| // | ||
| // Example – emit Langfuse session/user attributes alongside the standard ones: |
There was a problem hiding this comment.
[/tdd] Template expansion feature is documented but not implemented in runtime.
💡 Missing implementation
The documentation promises template variable expansion:
// Values support template variables using {{ variable }} syntax, where the
// variable name is any OTLP attribute key already computed for the span
// (e.g. {{ github.aw.episode.id }}, {{ github.actor }}).But send_otlp_span.cjs only has parseOTLPCustomAttributes and buildCustomOTLPAttributes — no expandOTLPAttributeTemplate function exists. The PR description claims this function was added, but it's missing from the implementation.
Impact: Users who configure templates like "{{ gh-aw.episode.id }}" will get literal strings in their OTLP spans instead of resolved values.
What's needed: Implement template expansion in buildCustomOTLPAttributes before merging:
function expandTemplate(template, attributes) {
return template.replace(/\{\{\s*([a-zA-Z0-9_.-]+)\s*\}\}/g, (match, varName) => {
const attr = attributes.find(a => a.key === varName);
if (!attr) return "";
if (attr.value.stringValue) return attr.value.stringValue;
if (attr.value.intValue !== undefined) return String(attr.value.intValue);
return "";
});
}| expect(buildCustomOTLPAttributes()).toEqual([]); | ||
| }); | ||
|
|
||
| it("returns static attribute values as-is", () => { |
There was a problem hiding this comment.
[/tdd] Test validates static values but doesn't test the advertised template expansion feature.
💡 Missing template expansion tests
This test verifies that static attribute values pass through as-is:
process.env.GH_AW_OTLP_ATTRIBUTES = JSON.stringify({
"langfuse.session.id": "my-session", // ← static value
"langfuse.user.id": "my-user",
});But the PR description and frontmatter documentation promise template variable expansion:
attributes:
langfuse.session.id: "{{ gh-aw.episode.id }}" # ← template, should expand
user.id: "{{ gh-aw.run.actor }}"What's missing: Tests that verify template expansion resolves variables from the span's existing attributes:
it("expands template variables from span attributes", () => {
process.env.GH_AW_OTLP_ATTRIBUTES = JSON.stringify({
"langfuse.session.id": "{{ gh-aw.episode.id }}",
"user.id": "{{ github.actor }}"
});
const spanAttributes = [
{ key: "gh-aw.episode.id", value: { stringValue: "ep-12345" } },
{ key: "github.actor", value: { stringValue: "pelikhan" } }
];
const result = buildCustomOTLPAttributes(spanAttributes);
expect(result).toContainEqual({ key: "langfuse.session.id", value: { stringValue: "ep-12345" } });
expect(result).toContainEqual({ key: "user.id", value: { stringValue: "pelikhan" } });
});| * | ||
| * @returns {Array<{key: string, value: object}>} | ||
| */ | ||
| function buildCustomOTLPAttributes() { |
There was a problem hiding this comment.
[/tdd] Function signature doesn't support template expansion as documented.
💡 Missing attributes parameter
buildCustomOTLPAttributes() is called after all standard span attributes are built:
attributes.push(...buildExperimentAttributes(experimentAssignments));
attributes.push(...buildEpisodeAttributesFromContext(awInfo, runId, runAttempt));
attributes.push(...buildCustomOTLPAttributes()); // ← called AFTER attributes are builtBut template expansion requires access to the already-computed attributes to resolve variables like {{ gh-aw.episode.id }}. The current signature function buildCustomOTLPAttributes() has no way to access them.
Fix: Change the signature to accept the existing attributes:
function buildCustomOTLPAttributes(existingAttributes = []) {
const customDefs = parseOTLPCustomAttributes();
if (!customDefs) return [];
const result = [];
for (const [key, value] of Object.entries(customDefs)) {
if (typeof key !== "string" || !key || typeof value !== "string") continue;
const expanded = expandTemplate(value, existingAttributes);
if (expanded !== "") {
result.push(buildAttr(key, expanded));
}
}
return result;
}Then update call sites:
attributes.push(...buildCustomOTLPAttributes(attributes));Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Done — reverted the |
🧪 Test Quality Sentinel Report
📊 Metrics & Test Classification (67 tests analyzed)
Test Classification DetailsGo Tests (pkg/workflow/)
Go Summary: 8/8 tests are design tests with proper table-driven structure, edge cases, and no mock libraries. JavaScript Tests (actions/setup/js/)Design tests (49): Most vitest tests with assertions verify observable outputs correctly. Implementation tests (10 with zero assertions):
Key JavaScript findings:
Test Inflation Analysis
Language SupportTests analyzed:
|
There was a problem hiding this comment.
✅ Test Quality Sentinel: 71/100. Test quality is acceptable — 14.9% of new tests are implementation tests (threshold: 30%). However, 10 JavaScript tests lack assertions and test inflation is high in send_otlp_span.test.cjs (4.43:1). See detailed comment for recommendations.
|
@copilot merge main and recompile |
…ibutes-support # Conflicts: # .github/workflows/daily-cache-strategy-analyzer.lock.yml # .github/workflows/daily-fact.lock.yml # .github/workflows/daily-observability-report.lock.yml # .github/workflows/duplicate-code-detector.lock.yml # .github/workflows/grumpy-reviewer.lock.yml # .github/workflows/issue-arborist.lock.yml # .github/workflows/necromancer.lock.yml # .github/workflows/schema-feature-coverage.lock.yml # .github/workflows/smoke-call-workflow.lock.yml # .github/workflows/smoke-codex.lock.yml Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Done — merged main, rebuilt the binary, and recompiled all 233 workflows (0 errors). |
Document the new observability.otlp.attributes frontmatter field introduced in #33846. Adds a reference row in the OpenTelemetry Fields table, a dedicated "Custom span attributes" section with an example using GitHub Actions expressions, a note about runner log masking, and a corresponding mention in the frontmatter reference. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Workflows have no way to attach custom OTLP attributes (e.g. Langfuse session/user tracking attributes) to spans. This adds a flexible, user-configurable
observability.otlp.attributesfrontmatter field with{{ variable }}template syntax resolved against already-computed span attributes.Frontmatter schema
Template variable names match
[a-zA-Z0-9_.-]— covering all standard gh-aw OTLP attribute keys. Unknown variables expand to empty string and are omitted from the span.Go (compiler)
frontmatter_types.go:Attributes map[string]stringadded toOTLPConfigobservability_otlp.go: encodes attributes asGH_AW_OTLP_ATTRIBUTESJSON env var (same pattern asGH_AW_OTLP_ENDPOINTS)compiler_orchestrator_workflow.go: fixed a latent bug where the observability merge for shared-workflow imports clobbered all fields exceptendpointimport_field_extractor.go:mergeObservabilityConfigsnow collects and merges custom attributes from imports (main workflow wins; first import wins among imports)JavaScript (runtime)
send_otlp_span.cjs:parseOTLPCustomAttributes/expandOTLPAttributeTemplate/buildCustomOTLPAttributes— reads env var and expands templates against the span's already-built attribute array; applied in bothsendJobSetupSpanandsendJobConclusionSpanemit_outcome_spans.cjs: custom attributes applied to the outcome summary spanpr-sous-chef: automated branch update from run https://github.com/github/gh-aw/actions/runs/26259944210