Smoke Test: OTEL Backends - 25953787817

[github-actions]

Overall Result: FAIL

The current run (25953787817) emitted one span to the local OTEL mirror with the correct github.run_id, but 2 OTLP export errors were recorded. Grafana Tempo is completely empty (no traces from any service). Sentry MCP tools only surface error events — span-level ingestion cannot be confirmed.

Run URL: https://github.com/github/gh-aw/actions/runs/25953787817

---

Checklist

• Local send verification — inconclusive (span found for this run; 2 export errors)
• Sentry query verification — inconclusive (project found, connection works; no span-search tool)
• Grafana query verification — fail (Tempo datasource reachable but completely empty)

---

Evidence

Step 1 — Local OTEL (send_status = inconclusive)

Check	Result
OTEL_EXPORTER_OTLP_ENDPOINT	set
OTEL_EXPORTER_OTLP_HEADERS	set
GH_AW_OTLP_ENDPOINTS	set
OTEL_SERVICE_NAME	gh-aw.smoke-otel-backends
/tmp/gh-aw/otel.jsonl exists	yes (1 line)
Span for run 25953787817	yes — gh-aw.agent.setup with run_id=25953787817
OTLP export error count	2

Spans reached the local file mirror but remote delivery is uncertain due to non-zero export errors.

Step 2 — Sentry (sentry_status = inconclusive)

Check	Result
MCP connection	connected (github.sentry.io, region us.sentry.io)
Organization	github found
Project	gh-aw (id 4511347087179777) found
Current-run spans	cannot confirm — no span-search tool available
Recent gh-aw error events (14d)	none found via list_events
Representative link	https://github.sentry.io/explore/discover/homepage/?project=4511347087179777&statsPeriod=14d

Sentry MCP only exposes error-event tools. The project exists and the connection is healthy, but OTLP trace ingestion cannot be confirmed.

Step 3 — Grafana Tempo (grafana_status = fail)

Check	Result
MCP connection	connected
Tempo datasource	grafanacloud-traces (uid grafanacloud-traces)
{resource.service.name="gh-aw"} last 24h	0 traces
{rootServiceName="gh-aw"}	0 traces
{duration>0} last 30 min (broad probe)	0 traces
tempo_get-attribute-names	only intrinsic attributes — no resource.* or span.* scope

The absence of any resource.* attribute scope confirms no OTLP data has ever been ingested into this Grafana Cloud Tempo instance, or retention has expired.

---

Blockers

1. OTLP export errors (emit-side)

2 errors in /tmp/gh-aw/otlp-export-errors.count. Likely root cause for backends not receiving spans. Investigate:

• Whether OTEL_EXPORTER_OTLP_ENDPOINT and GH_AW_OTLP_ENDPOINTS point to live, authenticated endpoints.
• HTTP response codes in the OTLP exporter log (401/403/5xx).

2. Grafana Tempo — no data ingested (Grafana read-side / emit-side)

Every query including {duration>0} returns empty. Either:

• OTLP endpoint for Grafana is misconfigured or unreachable.
• GH_AW_OTEL_GRAFANA_ENDPOINT / GH_AW_OTEL_GRAFANA_HEADERS secrets are stale or missing.
• Grafana Cloud stack mismatch (writing to a different stack than the one being queried).

3. Sentry — span-level visibility unverifiable (tooling gap)

The Sentry MCP server only exposes error-event tools. To close this gap:

• Add a direct Sentry Trace API call using GH_AW_OTEL_SENTRY_ENDPOINT + GH_AW_OTEL_SENTRY_AUTHORIZATION.
• Confirm whether Sentry is configured as an OTLP trace backend for this project.

Generated by 🧪 Smoke OTEL Backends for issue #32532 · ● 3.9M · ◷

• expires on May 16, 2026, 7:37 AM UTC

[github-actions]

This issue is being closed as outdated. A newer issue has been created: #32570

View newer issue

---

This action was performed automatically by the Smoke OTEL Backends workflow.