fix(otel): replace gen_ai.provider.name with gen_ai.system + engine-to-system mapping

[Copilot]

Agent spans were emitting the non-standard gen_ai.provider.name attribute with raw gh-aw engine IDs, causing OTel backends (Grafana, Datadog, Honeycomb) to treat them as plain INTERNAL spans rather than routing them to native GenAI dashboards.

Changes

• send_otlp_span.cjs: Adds a prototype-safe ENGINE_TO_SYSTEM_MAP (using Object.create(null)) and replaces gen_ai.provider.name with the standard gen_ai.system attribute on the agent span. Preserves the original engine ID in a new gh-aw.engine attribute.
• send_otlp_span.test.cjs: Updates assertions to match the new attribute names and mapped values. Adds a test for the unknown-engine fallback (raw engine ID passed through as-is).
• docs/src/content/docs/reference/frontmatter.md: Updates the agent span attributes table to document gen_ai.system and gh-aw.engine, and adds a migration note for users upgrading from gen_ai.provider.name.

Mapping

gh-aw engine	gen_ai.system
copilot	github_models
claude	anthropic
codex	openai
gemini	google_vertex_ai
(unknown)	(engine ID as-is)

Attribute diff on agent span

- "gen_ai.provider.name": "claude"
+ "gen_ai.system":        "anthropic"
+ "gh-aw.engine":         "claude"

[Copilot]

Pull request overview

Updates OTLP agent span attributes to align with OpenTelemetry GenAI semantic conventions so observability backends can correctly classify spans as GenAI.

Changes:

• Add an engine-id → gen_ai.system mapping table for known engines.
• Replace gen_ai.provider.name with standard gen_ai.system, while preserving the original engine ID as a custom attribute.
• Update Vitest assertions to reflect the new attribute names/values.

Show a summary per file

File	Description
actions/setup/js/send_otlp_span.cjs	Introduces ENGINE_TO_SYSTEM_MAP and emits gen_ai.system (mapped) plus a custom attribute for the raw engine ID on the agent span.
actions/setup/js/send_otlp_span.test.cjs	Updates tests to assert gen_ai.system and the preserved raw engine attribute instead of gen_ai.provider.name.

Copilot's findings

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (2)

actions/setup/js/send_otlp_span.cjs:38

• ENGINE_TO_SYSTEM_MAP is a plain object and the lookup uses ENGINE_TO_SYSTEM_MAP[engineId]. If engineId is set to a prototype property name like "constructor"/"__proto__", the lookup can return a non-string and end up emitting gen_ai.system as "[object Object]" or a function source string. Use a prototype-safe map (e.g., Object.create(null) for the mapping object, or Object.hasOwn(...) / hasOwnProperty checks before reading) and ensure the resolved value is a string.

const ENGINE_TO_SYSTEM_MAP = {
  copilot: "github_models",
  claude: "anthropic",
  codex: "openai",
  gemini: "google_vertex_ai",
};

actions/setup/js/send_otlp_span.cjs:940

• The agent span currently already includes gh-aw.engine.id via the shared attributes array, and this change adds a second, differently-named attribute gh-aw.engine. This duplicates the same value and diverges from the existing naming convention (gh-aw.engine.id) used elsewhere in this file for engine IDs. Consider reusing gh-aw.engine.id for the preserved engine ID (and only emitting gen_ai.system as the mapped value) to avoid redundant attributes and inconsistent field names.

    if (engineId) {
      const genAiSystem = ENGINE_TO_SYSTEM_MAP[engineId] || engineId;
      agentAttributes.push(buildAttr("gen_ai.system", genAiSystem));
      agentAttributes.push(buildAttr("gh-aw.engine", engineId));
    }

• Files reviewed: 2/2 changed files
• Comments generated: 2

[github-actions]

🧪 Test Quality Sentinel Report

Test Quality Score: 100/100

✅ Excellent test quality

Metric	Value
New/modified tests analyzed	2
✅ Design tests (behavioral contracts)	2 (100%)
⚠️ Implementation tests (low value)	0 (0%)
Tests with error/edge cases	2 (100%)
Duplicate test clusters	0
Test inflation detected	No (test +6 lines, production +25 lines → 0.24 ratio)
🚨 Coding-guideline violations	None

---

Test Classification Details

Test	File	Classification	Issues Detected
includes gen_ai.request.model, gen_ai.system, gh-aw.engine ...	actions/setup/js/send_otlp_span.test.cjs	✅ Design	None
omits gen_ai.request.model, gen_ai.system, gh-aw.engine ...	actions/setup/js/send_otlp_span.test.cjs	✅ Design	None

---

Analysis

Both modified tests verify observable OTLP span attributes — the exact key names and values that consumers of telemetry data would see. This is a textbook behavioral contract test:

• Test 1 confirms the new attribute mapping: gen_ai.system = "anthropic" (OTel semantic convention) and gh-aw.engine = "claude" (engine identity) replacing the deprecated gen_ai.provider.name.
• Test 2 covers the edge/absence case: when engine/model/workflow data is missing, neither gen_ai.system nor gh-aw.engine should appear in the span — preventing phantom attributes.

Mocking of fetch (via vi.fn()) is legitimate — it is external I/O, not an internal business-logic function.

---

Language Support

Tests analyzed:

• 🟨 JavaScript (*.test.cjs): 2 tests (vitest)

---

Verdict

✅ Check passed. 0% of new tests are implementation tests (threshold: 30%). Both tests enforce behavioral contracts on OTLP telemetry attribute semantics.

---

📖 Understanding Test Classifications

Design Tests (High Value) verify what the system does:

• Assert on observable outputs, return values, or state changes
• Cover error paths and boundary conditions
• Would catch a behavioral regression if deleted
• Remain valid even after internal refactoring

Implementation Tests (Low Value) verify how the system does it:

• Assert on internal function calls (mocking internals)
• Only test the happy path with typical inputs
• Break during legitimate refactoring even when behavior is correct
• Give false assurance: they pass even when the system is wrong

References: §25138992304

🧪 Test quality analysis by Test Quality Sentinel · ● 329.1K · ◷

[github-actions]

✅ Test Quality Sentinel: 100/100. Test quality is excellent — 0% of new tests are implementation tests (threshold: 30%). Both tests enforce behavioral contracts on OTLP span attribute semantics.

[pelikhan]

@copilot review all comments

[Copilot]

@copilot review all comments

Done in commit 3c60f42. Addressed all three review comments:

1. Prototype-safety: ENGINE_TO_SYSTEM_MAP now uses Object.assign(Object.create(null), {...}) to prevent prototype-pollution from keys like "constructor" or "__proto__".
2. Fallback test: Added a test case ("custom-engine") verifying that unknown engine IDs are passed through as-is for both gen_ai.system and gh-aw.engine.
3. Docs: Updated the agent span attributes table in frontmatter.md to document gen_ai.system (with examples of mapped values) and gh-aw.engine, plus added a migration note pointing out the change from gen_ai.provider.name.