You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This commit was created on GitHub.com and signed with GitHub’s verified signature.
Added
A chronicle-observer-quarantined counter records each time an observer is quarantined, so you can alert on observer quarantine as well as on partitions that ran out of retries. (#4427)
Changed
The observer failure counters (chronicle-observer-partitions-failed, chronicle-observer-partition-retry-attempts, chronicle-observer-partitions-quarantined) no longer carry a partition tag. That tag held the event source id, which could identify a person and created one series per failing partition. They now have one series per observer, tagged by event store, namespace, observer and event sequence.
Their exported names no longer carry their description as a unit suffix: in Prometheus they are chronicle_observer_partitions_failed_total, chronicle_observer_partition_retry_attempts_total and chronicle_observer_partitions_quarantined_total, and the description is now the metric's help text. Update dashboards and alert rules that use the old suffixed names; the rules in Get alerted when observers stop processing match both.
The counters' descriptions now say what they count: chronicle-observer-partitions-failed counts every failed attempt, including each retry, not failed partitions.