automated observability

---

title: Automated Observability category: concepts tags: [harness, observability, metrics, layer-5, quality] status: developing created: 2026-04-28 updated: 2026-04-28 sources:

• "harness-implementation-plan" related:
• "agentic-harness"
• "adversarial-verification"
• "persistent-memory" layer: "Layer_5" summary: Layer 5 enforces that every subtask output includes an ObservabilitySpec — metrics, health checks, and alert conditions. If require_metrics is enabled (default), the subtask is not complete until instrumentation code exists in the change. provenance: extracted: 0.85 inferred: 0.15 ambiguous: 0.0

---

Automated Observability

Origin Principle

Netflix's chaos engineering and Google's SRE principles: build operability in from the start, not as an afterthought. Observability is part of the definition of done.

Flow

subtask_verified (critics passed)
    ↓
AI call: extract metrics, health checks, alerts from code change + task spec
    ↓
Produce ObservabilitySpec
    ↓
If require_metrics=true (default):
  → Scan code changes for instrumentation matching MetricDefinitions
  → If missing: generate scaffolding, block until wired
    ↓
Store ObservabilitySpec with SubtaskResult
    ↓
subtask_observable → mark task as done

ObservabilitySpec Data Contract

• metrics — each with name, type (counter/gauge/histogram), description, unit, instrumentation_code
• health_checks — each with name, method, expected_result
• alert_conditions — each with metric_name, threshold, action

Every metric must answer "is this working correctly?" — no vanity metrics (like request_count without context).

Instrumentation Verification (Heuristic)

The system scans code changes for:

• metric.name as string literal
• metric.type usage patterns (counter/gauge/histogram)
• instrumentation_code pattern matching

If require_metrics=true: missing instrumentation blocks completion. If require_metrics=false: only a warning is logged.

Note: This is heuristic — it catches obvious omissions, not compile-time guarantees. require_metrics defaults to true because observability is definition-of-done. Setting it to false is a deliberate concession, not an opt-out of the layer.

Extension Interface

Type	Name	Description
Event consumed	subtask_verified	Enforce observability for verified subtask
Event emitted	subtask_observable	Task marked as done
Event emitted	observability_missing	Missing metrics flagged
Tool	define-observability	Generate metrics spec for a task
Tool	verify-observability	Check which metrics have instrumentation
Command	/harness-observability-status	Metrics defined vs wired

Config

{
  "observability": {
    "require_metrics": true
  }
}

Files

• lib/harness-observability.ts — ObservabilityEnforcer class
• extensions/harness-observability.ts — Extension with enforcement hooks