Skip to content

Phase 2 Service Onboarding

Jump to bottom
Ivan P edited this page Jun 5, 2026 · 5 revisions

OTel auto-instrumentation for all 8 services. Each service has two issues: code changes (instrumentation library setup) and Helm chart changes (environment variable injection). Services are independent and can be done in any order.

Total estimated effort: ~25h across all 8 services Recommended starting points: acapy-agent and traction

For detailed per-service technical specs see report/index.html → Repository Telemetry Assessments.

acapy-agent

Issue: acapy-agent — OTel Instrumentation (Code)

Labels: phase-2 instrumentation python acapy Estimated Effort: ~2.5h

Description: Add OpenTelemetry auto-instrumentation to acapy-agent to enable distributed tracing for HTTP endpoints and PostgreSQL queries.

Requirements:

  • Add opentelemetry-api, opentelemetry-sdk, opentelemetry-exporter-otlp, opentelemetry-instrumentation-aiohttp-server to dependencies
  • Initialize OTel SDK in acapy_agent/__main__.py
  • Add aiohttp server middleware in acapy_agent/admin/server.py
  • Add PostgreSQL tracing via opentelemetry-instrumentation-psycopg (psycopg v3)

Acceptance Criteria:

  • OTel SDK initializes on startup without errors
  • Inbound HTTP requests to the admin API generate trace spans visible in Tempo
  • PostgreSQL queries appear as child spans within HTTP request traces
  • No regression in existing test suite

Issue: acapy-agent — Helm Chart Updates

Labels: phase-2 helm acapy Estimated Effort: ~30min

Description: Inject OTel environment variables into the acapy-agent deployment via Helm chart.

Requirements:

  • Add OTEL_SERVICE_NAME, OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_PYTHON_LOGGING_AUTO_INSTRUMENTATION_ENABLED to the deployment env block
  • Values should be configurable via values.yaml with sensible defaults

Acceptance Criteria:

  • Env vars present in deployed pod (kubectl exec verify)
  • Traces appear in Tempo after deployment
  • Helm chart change reviewed and merged

acapy-vc-authn-oidc

Issue: acapy-vc-authn-oidc — OTel Instrumentation (Code)

Labels: phase-2 instrumentation python vc-authn Estimated Effort: ~2h

Description: Add OTel auto-instrumentation to the FastAPI backend with MongoDB and Redis driver tracing.

Requirements:

  • Add opentelemetry-instrumentation-fastapi, opentelemetry-instrumentation-httpx, opentelemetry-instrumentation-pymongo, opentelemetry-instrumentation-redis
  • Initialize OTel SDK in oidc-controller/api/main.py
  • Apply FastAPIInstrumentor.instrument_app(app)

Acceptance Criteria:

  • FastAPI request spans visible in Tempo
  • MongoDB and Redis operations appear as child spans
  • No regression in existing test suite

Issue: acapy-vc-authn-oidc — Helm Chart Updates

Labels: phase-2 helm vc-authn Estimated Effort: ~30min

Description: Inject OTEL_SERVICE_NAME and OTEL_EXPORTER_OTLP_ENDPOINT into the deployment.

Acceptance Criteria:

  • Env vars present in deployed pod
  • Traces appear in Tempo after deployment

traction

Issue: traction — OTel Instrumentation (Code)

Labels: phase-2 instrumentation nodejs python traction Estimated Effort: ~3h

Description: Add OTel instrumentation to both the tenant-ui (Node.js/Express) and the ACA-Py plugin (Python/aiohttp) components.

Requirements:

  • tenant-ui: Add @opentelemetry/sdk-node, @opentelemetry/auto-instrumentations-node; create src/tracing.ts; import before Express starts in src/index.ts
  • ACA-Py plugin: Wrap startup command with opentelemetry-instrument CLI in plugins/docker/Dockerfile; add opentelemetry-instrumentation-asyncpg for PostgreSQL

Acceptance Criteria:

  • tenant-ui HTTP spans visible in Tempo
  • ACA-Py plugin spans visible in Tempo, linked to tenant-ui parent spans
  • PostgreSQL spans appear as children of ACA-Py spans
  • No regression in existing test suite

Issue: traction — Helm Chart Updates

Labels: phase-2 helm traction Estimated Effort: ~30min

Description: Inject OTel env vars into both the tenant-ui and ACA-Py plugin deployments.

Requirements:

  • OTEL_SERVICE_NAME and OTEL_EXPORTER_OTLP_ENDPOINT for both deployments
  • Values configurable via values.yaml

Acceptance Criteria:

  • Env vars present in both deployed pods
  • Traces from both components visible and linked in Tempo

acapy-endorser-service

Issue: acapy-endorser-service — OTel Instrumentation (Code)

Labels: phase-2 instrumentation python endorser Estimated Effort: ~2h

Description: Add OTel auto-instrumentation to the FastAPI endorser service with asyncpg and SQLAlchemy tracing.

Requirements:

  • Add opentelemetry-instrumentation-fastapi, opentelemetry-instrumentation-aiohttp-client, opentelemetry-instrumentation-asyncpg, opentelemetry-instrumentation-sqlalchemy
  • Initialize OTel SDK in endorser/api/main.py

Acceptance Criteria:

  • FastAPI request spans visible in Tempo
  • Database query spans (asyncpg / SQLAlchemy) appear as children
  • No regression in existing test suite

Issue: acapy-endorser-service — Helm Chart Updates

Labels: phase-2 helm endorser Estimated Effort: ~30min

Description: Inject OTEL_SERVICE_NAME and OTEL_EXPORTER_OTLP_ENDPOINT into the endorser deployment.

Acceptance Criteria:

  • Env vars present in deployed pod
  • Traces appear in Tempo after deployment

didwebvh-server-py

Issue: didwebvh-server-py — OTel Instrumentation (Code)

Labels: phase-2 instrumentation python didwebvh Estimated Effort: ~5h

Description: Add OTel auto-instrumentation to the didwebvh FastAPI server including centralized OTel init and SQLAlchemy database tracing.

Requirements:

  • Add opentelemetry-instrumentation-fastapi, opentelemetry-instrumentation-logging, opentelemetry-instrumentation-sqlalchemy to server/pyproject.toml
  • Create server/app/otel.py for centralized TracerProvider + BatchSpanProcessor + OTLPExporter setup
  • Call OTel init from server/main.py before uvicorn.run
  • Apply FastAPIInstrumentor.instrument_app(app) in server/app/__init__.py
  • Apply SQLAlchemyInstrumentor().instrument(engine=self.engine) in server/app/plugins/storage.py

Acceptance Criteria:

  • FastAPI request spans visible in Tempo
  • SQLAlchemy/psycopg2 query spans appear as children
  • Log correlation enabled (OTEL_PYTHON_LOG_CORRELATION=true)
  • No regression in existing test suite

Issue: didwebvh-server-py — Helm Chart Updates

Labels: phase-2 helm didwebvh Estimated Effort: ~30min

Description: Inject OTel env vars into the didwebvh-server-py deployment.

Requirements:

  • OTEL_SERVICE_NAME, OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_PYTHON_LOG_CORRELATION

Acceptance Criteria:

  • Env vars present in deployed pod
  • Traces appear in Tempo after deployment

credo-ts

Issue: credo-ts — OTel Instrumentation (Code)

Labels: phase-2 instrumentation typescript credo Estimated Effort: ~2h

Description: Add OTel instrumentation via a preloader bootstrap wrapper for the credo-ts library and its consuming services.

Requirements:

  • Add @opentelemetry/sdk-node, @opentelemetry/api, @opentelemetry/auto-instrumentations-node, @opentelemetry/exporter-trace-otlp-grpc
  • Create otel-wrapper.ts bootstrap preloader
  • Initialize in the client agent constructor in packages/core/src/agent/Agent.ts
  • Add @opentelemetry/instrumentation-pg for PostgreSQL or @opentelemetry/instrumentation-sqlite3 for SQLite

Note: credo-ts is a library — Helm chart changes target the consuming service's chart.

Acceptance Criteria:

  • OTel spans generated for outbound HTTP calls and DB queries
  • Traces visible in Tempo from a consuming service
  • No regression in existing test suite

Issue: credo-ts consuming service — Helm Chart Updates

Labels: phase-2 helm credo Estimated Effort: ~30min

Description: Inject OTel env vars into the consuming service's deployment (credo-ts is a library; env vars go on the host service chart).

Requirements:

  • OTEL_SERVICE_NAME and OTEL_EXPORTER_OTLP_ENDPOINT on the consuming service deployment

Acceptance Criteria:

  • Env vars present in consuming service pod
  • Traces from credo-ts operations visible in Tempo

didcomm-mediator-credo

Issue: didcomm-mediator-credo — OTel Instrumentation (Code)

Labels: phase-2 instrumentation typescript mediator Estimated Effort: ~2.5h

Description: Resolve the existing Dockerfile OTel exporter build issue and add OTel instrumentation via a preloader file.

Requirements:

  • Resolve Dockerfile build failure: add @opentelemetry/exporter-trace-otlp-http and @opentelemetry/exporter-metrics-otlp-http in the compilation phase
  • Create/update apps/mediator/instrumentation.js as the Node.js preloader
  • Enable enhancedDatabaseReporting: true for @opentelemetry/instrumentation-pg to capture queue operations

Acceptance Criteria:

  • Docker image builds without OTel exporter errors
  • Mediator HTTP and pg spans visible in Tempo
  • No regression in existing behaviour

Issue: didcomm-mediator-credo — Helm Chart Updates

Labels: phase-2 helm mediator Estimated Effort: ~30min

Description: Inject OTel env vars into the mediator deployment.

Requirements:

  • OTEL_SERVICE_NAME, OTEL_EXPORTER_OTLP_ENDPOINT (port 4318 HTTP for this service)

Acceptance Criteria:

  • Env vars present in deployed pod
  • Traces appear in Tempo after deployment

bc-wallet-demo

Issue: bc-wallet-demo — OTel Instrumentation (Code)

Labels: phase-2 instrumentation typescript bc-wallet Estimated Effort: ~6h

Description: Add OTel auto-instrumentation to the bc-wallet-demo server with MongoDB/Mongoose database tracing.

Requirements:

  • Create server/src/instrumentation.ts using NodeSDK with getNodeAutoInstrumentations()
  • Import as first line of server/src/index.ts (or use --require flag in start script)
  • Add @opentelemetry/instrumentation-mongoose and @opentelemetry/instrumentation-mongodb for MongoDB

Acceptance Criteria:

  • Server HTTP spans visible in Tempo
  • MongoDB/Mongoose operations appear as child spans
  • No regression in existing test suite

Issue: bc-wallet-demo — Helm Chart Updates

Labels: phase-2 helm bc-wallet Estimated Effort: ~30min

Description: Inject OTel env vars into the bc-wallet-demo server deployment.

Requirements:

  • OTEL_SERVICE_NAME, OTEL_EXPORTER_OTLP_ENDPOINT, OTEL_NODEJS_AUTO_INSTRUMENTATIONS_ENABLED

Acceptance Criteria:

  • Env vars present in deployed pod
  • Traces appear in Tempo after deployment

Clone this wiki locally