Skip to content

fix: align OTEL attributes with gen_ai semconv spec#3488

Merged
lpcox merged 3 commits into
mainfrom
fix/otel-smoke-build-local
May 20, 2026
Merged

fix: align OTEL attributes with gen_ai semconv spec#3488
lpcox merged 3 commits into
mainfrom
fix/otel-smoke-build-local

Conversation

@lpcox
Copy link
Copy Markdown
Collaborator

@lpcox lpcox commented May 20, 2026

Aligns api-proxy OTEL span attributes with the OpenTelemetry GenAI semantic conventions:

Renamed attributes:

  • awf.providergen_ai.provider.name
  • gen_ai.system → removed (deprecated in spec)
  • awf.cache_read_tokensgen_ai.usage.cache_read.input_tokens
  • awf.cache_write_tokensgen_ai.usage.cache_creation.input_tokens
  • awf.streaminggen_ai.request.stream

New standard attributes:

  • gen_ai.operation.name (set to chat)
  • gen_ai.usage.reasoning.output_tokens

Tests updated to match.

lpcox and others added 3 commits May 20, 2026 16:17
… code

The workflow was using --skip-pull with pre-built GHCR images (v0.25.29)
which don't include the recently-merged otel.js module. Switch to
--build-local so the api-proxy container is built from source and
includes the OTEL tracing implementation.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Streaming OpenAI/Copilot API responses only include usage data in the
final SSE chunk when the client sends stream_options: {include_usage: true}.
Without this, the token tracker never extracts usage and OTEL spans have
no gen_ai.usage.* attributes.

The api-proxy now injects this option automatically for OpenAI-compatible
providers (copilot, openai, opencode) when stream: true is set and
stream_options is not already present.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Rename attributes to match the OpenTelemetry GenAI semantic conventions:
- awf.provider → gen_ai.provider.name
- gen_ai.system → removed (deprecated)
- awf.cache_read_tokens → gen_ai.usage.cache_read.input_tokens
- awf.cache_write_tokens → gen_ai.usage.cache_creation.input_tokens
- awf.streaming → gen_ai.request.stream

Add new standard attributes:
- gen_ai.operation.name (set to 'chat')
- gen_ai.usage.reasoning.output_tokens

Reference: https://opentelemetry.io/docs/specs/semconv/registry/attributes/gen-ai/

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings May 20, 2026 23:54
@lpcox lpcox merged commit f071abb into main May 20, 2026
35 of 36 checks passed
@lpcox lpcox deleted the fix/otel-smoke-build-local branch May 20, 2026 23:55
@github-actions
Copy link
Copy Markdown
Contributor

✅ Coverage Check Passed

Overall Coverage

Metric Base PR Delta
Lines 95.73% 95.80% 📈 +0.07%
Statements 95.56% 95.63% 📈 +0.07%
Functions 96.86% 96.86% ➡️ +0.00%
Branches 89.30% 89.34% 📈 +0.04%
📁 Per-file Coverage Changes (1 files)
File Lines (Before → After) Statements (Before → After)
src/config-writer.ts 83.0% → 85.6% (+2.54%) 83.0% → 85.6% (+2.54%)

Coverage comparison generated by scripts/ci/compare-coverage.ts

@github-actions
Copy link
Copy Markdown
Contributor

Smoke Test Results

GitHub API - 2 PR entries verified in recent-prs.json
Playwright - GitHub page title contains "GitHub"
File Verify - smoke-test file exists

Result: PASS

💥 [THE END] — Illustrated by Smoke Claude

@github-actions
Copy link
Copy Markdown
Contributor

🔥 Smoke Test: Copilot BYOK Offline Mode

Test Result
GitHub MCP connectivity ✅ PR listed: "fix: align OTEL attributes with gen_ai semconv spec"
GitHub.com HTTP ⚠️ Pre-step data unavailable (template not expanded)
File write/read ⚠️ Pre-step data unavailable (template not expanded)
BYOK inference (this response) ✅ Agent responding via api-proxy → api.githubcopilot.com

Running in BYOK offline mode (COPILOT_OFFLINE=true) via api-proxy → api.githubcopilot.com.

Author: @lpcox | Overall: PARTIAL (BYOK path confirmed ✅; pre-step outputs not expanded)

🔑 BYOK report filed by Smoke Copilot BYOK

@github-actions
Copy link
Copy Markdown
Contributor

🔬 Smoke Test Results

Test Result
GitHub MCP ✅ PR: "fix: align OTEL attributes with gen_ai semconv spec"
GitHub.com connectivity ✅ Reachable (MCP confirmed)
File write/read /tmp/gh-aw/agent/smoke-test-copilot-26196872954.txt verified

Overall: PASS

Author: @lpcox

📰 BREAKING: Report filed by Smoke Copilot

@github-actions
Copy link
Copy Markdown
Contributor

Smoke test: FAIL
✅ Merged PRs: fix: align OTEL attributes with gen_ai semconv spec; fix: Use --build-local in smoke-otel-tracing for latest api-proxy code
❌ safeinputs-gh: command unavailable; gh query returned PR titles
✅ Playwright: title contains GitHub
❌ Tavily: no search tools exposed
✅ File/bash, discussion comment, and npm ci && npm run build passed
Overall status: FAIL

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • registry.npmjs.org

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "registry.npmjs.org"

See Network Configuration for more information.

🔮 The oracle has spoken through Smoke Codex

@github-actions
Copy link
Copy Markdown
Contributor

🏗️ Build Test Suite Results

Ecosystem Project Build/Install Tests Status
Bun elysia 1/1 passed ✅ PASS
Bun hono 1/1 passed ✅ PASS
C++ fmt N/A ✅ PASS
C++ json N/A ✅ PASS
Deno oak N/A 1/1 passed ✅ PASS
Deno std N/A 1/1 passed ✅ PASS
.NET hello-world N/A ✅ PASS
.NET json-parse N/A ✅ PASS
Go color 1/1 passed ✅ PASS
Go env 1/1 passed ✅ PASS
Go uuid 1/1 passed ✅ PASS
Java gson 1/1 passed ✅ PASS
Java caffeine 1/1 passed ✅ PASS
Node.js clsx All passed ✅ PASS
Node.js execa All passed ✅ PASS
Node.js p-limit All passed ✅ PASS
Rust fd 1/1 passed ✅ PASS
Rust zoxide 1/1 passed ✅ PASS

Overall: 8/8 ecosystems passed — ✅ PASS

Generated by Build Test Suite for issue #3488 · ● 3.9M ·

@github-actions
Copy link
Copy Markdown
Contributor

Smoke Test: API Proxy OTEL Tracing Results

Scenario Status Notes
1. Module Loading otel.js loads successfully; exports: startRequestSpan, setTokenAttributes, endSpan, endSpanError, shutdown, isEnabled
2. Test Suite 32/32 tests passed (otel.test.js) — spans, token attrs, parent context, OTLP export, file fallback
3. Env Var Forwarding api-proxy-service.ts forwards OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_EXPORTER_OTLP_HEADERS, OTEL_SERVICE_NAME, GITHUB_AW_OTEL_TRACE_ID, GITHUB_AW_OTEL_PARENT_SPAN_ID
4. Token Tracker Integration onUsage callback exists in token-tracker-http.js (line 232) — OTEL hook point confirmed
5. OTEL Diagnostics No live spans exported (OTLP endpoint not configured); graceful degradation to file fallback confirmed

All 5 scenarios pass. OTEL tracing integration is fully functional.

📡 OTel tracing validated by Smoke OTel Tracing

@github-actions
Copy link
Copy Markdown
Contributor

Chroot Version Comparison — Smoke Test Results

Runtime Host Version Chroot Version Match?
Python Python 3.12.13 Python 3.12.3 ❌ NO
Node.js v24.15.0 v20.20.2 ❌ NO
Go go1.22.12 go1.22.12 ✅ YES

Result: ❌ Not all tests passed — Python and Node.js versions differ between host and chroot environment.

Tested by Smoke Chroot

@github-actions
Copy link
Copy Markdown
Contributor

Smoke Test: Gemini Engine Validation. Results: MCP: FAIL, Connectivity: FAIL, File Writing: PASS, Bash: PASS. Overall: FAIL.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • localhost

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "localhost"

See Network Configuration for more information.

💎 Faceted by Smoke Gemini

@github-actions
Copy link
Copy Markdown
Contributor

Smoke Test Results — Services Connectivity

Check Result
Redis PING ❌ FAIL (port 6379 blocked by AWF iptables)
PostgreSQL pg_isready ❌ FAIL (port 5432 blocked by AWF iptables)
PostgreSQL SELECT 1 ❌ FAIL (port 5432 blocked by AWF iptables)

Overall: FAIL

host.docker.internal resolves correctly to 172.17.0.1, but the AWF sandbox blocks non-HTTP ports (6379/5432) via iptables rules in setup-iptables.sh. Service containers are unreachable from inside the AWF agent container.

🔌 Service connectivity validated by Smoke Services

@lpcox lpcox review requested due to automatic review settings May 21, 2026 00:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant