Observability Baseline: Validation and Trust Scoring Telemetry

[beonde]

Context

Current CapiscIO validation and trust scoring pipeline lacks unified observability. We need minimal, language-agnostic telemetry across CLI, middleware, and API components to trace trust evaluation outcomes and detect scoring drift.

Objective

Establish a baseline observability layer for validation workflows, ensuring every trust computation and validation step can be traced, measured, and correlated across runtimes.

Scope

• Structured Logging — Implement pino (TypeScript) and structlog (Python) for stage-based logging.
• Metrics — Add OpenTelemetry counters for success/failure rates and latency per validation step.
• Traceability — Include unique validation_id across CLI, API, and middleware.
• Storage — Stream structured metrics to Honeycomb or OpenTelemetry Collector.

Deliverables

• Instrumentation in a2a-security middleware
• Shared OpenTelemetry schema (trust_score_v1.1.0 correlation)
• JSON log enrichment for CLI validator
• Initial dashboard (validation volume, failure %, latency)

Expected Outcomes

✅ End-to-end visibility into validation reliability.
✅ Identifiable cause of trust scoring discrepancies.
✅ Metrics to inform regression tests and SLA enforcement.

Dependencies

• ADR-004 (Canonical Trust Scoring Model)
• RFC-0002 (Trust Scoring Specification)

Status

proposed

---

Labels: observability, architecture, telemetry, decision-pending
Owner: Engineering Lead AI