Add structured observability following the metrics-refactor tech spec,
covering infrastructure, service base class, and application layers.
Layer 1 - Infrastructure:
- Add tg_ namespace prefix to all metrics (chunker, metering, LLM,
image-to-text, tool services)
- Remove dead SubscriberMetrics class
- Add instrument=False for config consumers to exclude from work metrics
- Add config_version gauge to ProcessorMetrics
- Separate TooManyRequests from error counters in ReceiverPool
- Add centrally defined histogram bucket presets (BUCKETS_STANDARD,
BUCKETS_LLM, BUCKETS_SESSION)
Layer 2 - Service base classes:
- Add DownstreamMetrics (duration, timeout, error) in
RequestResponseClient, wired via request_response_spec
- Add embeddings service metrics (request count, duration, batch size)
- Add reranker service metrics (request count, duration, result count)
- Add triples/graph-embeddings/document-embeddings query service metrics
(duration, result count)
- Add store write metrics to all three store base classes (duration,
batch size, error count), covering all 10 backend implementations
Layer 3 - Application:
- Agent orchestration: session outcomes, iteration count, per-tool
invocation/duration/error, LLM reasoning step duration
- Gateway dispatcher: per-service request count and duration
- Gateway auth: signing key state, known workspaces gauge, auth failure
reasons, authz decision outcomes
- Gateway config receiver: active flow count gauge
- Knowledge extraction pipeline: shared extraction metrics (duration,
triple count, empty-chunk) across definitions, relationships, and
ontology extractors with extractor label; ontology-specific metrics
(loaded count, element count, selection count/duration, no-match)
- IAM service: per-operation request count and duration, user/workspace
inventory gauges, API key lifecycle counters
Also adds tech spec and rewrites unit tests for the new metric classes.
Metrics guide