feat(den): add configurable observability

[OmarMcAdam]

Summary

• add mutually exclusive none, OpenTelemetry, and Sentry observability backends for den-api and den-web
• export correlated OTLP HTTP/protobuf traces, logs, and Hono metrics with shared/per-signal endpoint configuration
• add Helm values/templates/docs, secure BuildKit source-map uploads, and a pinned Grafana LGTM validation stack
• migrate Den runtime logging to structured, redacted logs with trace and request correlation

Validation

• pnpm --filter @openwork-ee/den-api exec bun test test/audit-events.test.ts test/observability.test.ts test/sentry-sourcemaps-build.test.ts — 28 passed
• pnpm --filter @openwork-ee/den-web test — 23 passed
• pnpm --filter @openwork-ee/utils test — 9 passed
• den-api and den-web typechecks and production builds passed
• pnpm install --frozen-lockfile passed
• Helm observability rendering tests and helm lint passed
• live Docker LGTM validation passed: connected den-web → den-api trace, correlated logs from both services, Hono duration/active metrics, redaction, and live none-mode fallback

Proof

• Fraimz: evals/results/2026-07-13T08-28-05-261Z/fraimz.html
• Result: Passed — 1 passed, 0 failed, 0 skipped

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
openwork-app	Ready	Preview, Comment	Jul 13, 2026 1:45pm
openwork-den	Ready	Preview, Comment	Jul 13, 2026 1:45pm
openwork-den-worker-proxy	Ready	Preview, Comment	Jul 13, 2026 1:45pm
openwork-landing	Ready	Preview, Comment, Open in v0	Jul 13, 2026 1:45pm

[OmarMcAdam]

fraimz — ✅ PASSED

1 passed · 0 failed · 0 skipped — run 2026-07-13T08-56-41-091Z

Full frame proof with validated screenshots: evals/results/2026-07-13T08-56-41-091Z/fraimz.html (re-run: pnpm fraimz --flow otel-hono)

✅ otel-hono — Den services provide configurable, correlated observability without app UI

Internal demo

1. The OTEL backend uses one collector config while preserving service.name for den-web and den-api

   🎙 I start den-web and den-api with the OTEL backend and one shared collector URL. Both services become healthy, while Grafana shows them as distinct den-web and den-api services.

   • ✅ The live merged Docker validator passes
   • ✅ The validator emits a passing JSON report
   • ✅ Tempo stores den-web and den-api resources in one connected trace
2. A Den web proxy request carries W3C trace context into Den API, whose logs use safe route labels

   🎙 I make a real request through den-web’s proxy into den-api. Grafana displays one connected trace spanning the Next.js server and the normalized Hono route, with status and request ID but no sensitive headers or query data.

   • ✅ The real den-web proxied OpenAPI request returns HTTP 200
   • ✅ The stored Hono span uses the normalized /openapi.json route
   • ✅ The stored Hono span carries request.id
   • ✅ The request secret is absent from stored trace and log output
3. Structured runtime logs include trace/span IDs and the OTLP log sink emits in the active trace context

   🎙 I open the logs view and see structured runtime logs from both services. Each request log carries its trace and span IDs, letting me jump from a log directly to the related trace.

   • ✅ Loki stores the den-web proxy completion log
   • ✅ The den-web OTLP log trace_id matches the connected Tempo trace
   • ✅ The den-web OTLP log has a span_id belonging to the den-web trace resource
   • ✅ Loki stores the den-api request-completion log
   • ✅ The den-api OTLP log trace_id matches the connected Tempo trace
   • ✅ The den-api OTLP log has a span_id belonging to the den-api trace resource
   • ✅ The validator marks den-web trace/span correlation as observed
   • ✅ The validator marks den-api trace/span correlation as observed
4. Den API uses Hono OTEL request metrics while traces, metrics, and logs can be disabled independently

   🎙 I open metrics and see Hono request duration and active-request measurements for den-api. Signal-specific environment controls let me disable logs, metrics, or traces independently without affecting the application.

   • ✅ Prometheus stores the duration metric for den-api
   • ✅ The Hono duration metric uses the normalized /openapi.json route label
   • ✅ The observed series is the Hono HTTP server request duration metric
   • ✅ Prometheus stores the active-request metric series for den-api, even when the current value is zero
   • ✅ The observed active-request series is the Hono HTTP server active-requests metric
   • ✅ The validator marks the Hono active-request metric series as observed
   • ✅ The per-signal exporter control test passes
   • ✅ The live den-web and den-api logs contain no unhandled runtime errors
5. With DEN_OBSERVABILITY_BACKEND=none, no exporter initializes and logs remain structured JSON stdout

   🎙 I run with observability disabled and the same application flow still works. Runtime logs fall back to structured JSON stdout, and no collector connection is attempted.

   • ✅ The live none-backend proxy request returns HTTP 200
   • ✅ The live none-backend run emits structured JSON stdout request logs for den-web and den-api
   • ✅ The live none-backend run shows no exporter connection or error attempt
   • ✅ The none-backend and JSON stdout tests pass
   • ✅ The none-backend command exercised config defaults and JSON stdout redaction
   • ✅ The none backend source check proves no OTEL/Sentry exporter startup path is entered
6. Mocked Sentry tests prove backend exclusivity and source-map gating without claiming live Sentry delivery

   🎙 I select Sentry instead of OTEL and the services initialize only Sentry, never both providers. Errors, traces, and logs are captured when a DSN is configured, and production builds upload source maps only when build credentials are present.

   • ✅ The Sentry exclusivity and source-map gating tests pass
   • ✅ The Sentry command covered single-initializer dispatch, credential redaction, and source-map upload gating
   • ✅ The source check proves Sentry/OTEL runtime exclusivity and source-map gating branches
   • ✅ Script frame 1 narrated: "I start den-web and den-api with the OTEL backend and one shared collector URL. Both ser"
   • ✅ Script frame 2 narrated: "I make a real request through den-web’s proxy into den-api. Grafana displays one connect"
   • ✅ Script frame 3 narrated: "I open the logs view and see structured runtime logs from both services. Each request lo"
   • ✅ Script frame 4 narrated: "I open metrics and see Hono request duration and active-request measurements for den-api"
   • ✅ Script frame 5 narrated: "I run with observability disabled and the same application flow still works. Runtime log"
   • ✅ Script frame 6 narrated: "I select Sentry instead of OTEL and the services initialize only Sentry, never both prov"

[cloudflare-workers-and-pages]

Deploying openwork with    Cloudflare Pages

Latest commit:	c14d296
Status:	✅  Deploy successful!
Preview URL:	https://0c7b87f8.openwork.pages.dev
Branch Preview URL:	https://otel-hono.openwork.pages.dev

View logs