RFC: High-rate runtime telemetry and status-write offload path

[lanycrost]

Summary

High-rate runtime events currently flow directly into Kubernetes status updates while Bobravoz also emits hot-path, high-cardinality metrics. The platform needs an explicit boundary between durable workflow state, operator-facing summaries, and operational telemetry before ingress/runtime workloads scale further.

Why this needs a discussion

This is bigger than a single SDK optimization. bobrapet already makes StepRun.status a bounded summary surface, while the SDK and Bobravoz still write high-rate activity directly into CRD status and default metrics. That is an ecosystem contract problem, not just a local performance bug.

Current evidence

• bubu-sdk-go/signals.go, effects.go, and k8s/impulse_stats.go patch StepRun or Impulse status on each runtime event.
• bubu-sdk-go#66 already tracks the impulse-trigger-stats slice of this write-amplification problem.
• bobravoz-grpc/pkg/metrics/hub_metrics.go and connector_metrics.go use high-cardinality labels such as StoryRun and binding identifiers on hot metrics.
• bobrapet already gives us controller-owned status on StoryTrigger, while StepRun.status.signals / signalEvents / effects are explicitly bounded small surfaces instead of a scalable event log.
• The current platform mixes three concerns on the same hot paths: controller decisions, operator-facing summaries, and high-rate telemetry counters.

Proposed decisions

1. StepRun.status and Impulse.status are summary surfaces only. They are not the sink for high-rate runtime telemetry.
2. Synchronous status writes remain only for controller-critical decisions and small operator summaries. High-rate trigger/signal/effect activity moves off the hot CRD path.
3. The preferred offload target is storage-backed runtime history / outbox data, not CRD-per-event resources. Metrics are for non-durable counters only.
4. Bobravoz default telemetry must drop StoryRun/binding high-cardinality labels; detailed labels become an explicit opt-in debugging mode.
5. Because the ecosystem is pre-release, we should migrate current CLI/UI/template consumers to the cleaner status/history split now instead of preserving the current write-amplified behavior for compatibility.

Cross-repo scope

• bobrapet: owns CRD/status boundaries plus any future history/outbox resource or controller-facing summary contract.
• bubu-sdk-go: must batch/offload runtime events instead of patching status on every signal/effect/trigger.
• bobravoz-grpc: must narrow default metric cardinality and align ingress/runtime telemetry to the same boundary.

Desired outcome

A cross-repo decision on runtime telemetry boundaries so follow-up issues can split cleanly across bubu-sdk-go, bobrapet, and Bobravoz without re-arguing the scaling model in each repo.