Skip to content

[Subscription] Fix consensus subscription metrics across Regions#18277

Merged
jt2594838 merged 1 commit into
apache:masterfrom
Caideyipi:fix/consensus-subscription-metrics-region
Jul 22, 2026
Merged

[Subscription] Fix consensus subscription metrics across Regions#18277
jt2594838 merged 1 commit into
apache:masterfrom
Caideyipi:fix/consensus-subscription-metrics-region

Conversation

@Caideyipi

Copy link
Copy Markdown
Collaborator

Description

Isolate consensus subscription metrics by Region

Consensus subscription queues from different Regions can share the same consumer-group/topic queue ID. The metrics singleton previously keyed queues and rates only by that ID, so one Region overwrote or hid the others.

This change keys metric state by (queue ID, Region ID), adds the region tag to every consensus-subscription metric, passes the Region when marking transfer rates, and removes only the matching Region's gauges and rate. Metric lifecycle operations are serialized and stale queue instances cannot deregister the active instance for the same key.

Make catch-up status observable and accurate

  • Add subscription_consensus_active, subscription_consensus_initialized, and subscription_consensus_seek_generation.
  • Keep subscription_current_commit_id for dashboard compatibility while exposing its actual seek-generation meaning under the new metric name.
  • Include entries waiting in the linger batch in subscription_consensus_lag, so lag cannot report zero while accepted data is still waiting to be emitted.
  • Remove stale per-Region Rate objects during deregistration.

For a stopped workload, an initialized active Region with zero lag now means the queue has no buffered, in-flight, pending, lingered, or unread WAL work.

Verification

  • mvn -Ddevelocity.off=true spotless:apply -pl iotdb-core/node-commons,iotdb-core/datanode
  • mvn -Ddevelocity.off=true checkstyle:check -pl iotdb-core/node-commons,iotdb-core/datanode
  • Targeted unit tests: 12 tests, 0 failures/errors
    • ConsensusSubscriptionPrefetchingQueueMetricsTest
    • ConsensusPrefetchingQueueTest
  • English and Chinese full-reactor test-compile were attempted. Both stop before the changed modules because generated iotdb-thrift-commons/.../TaskType.java cannot resolve javax.annotation in the local environment.

This PR has:

  • been self-reviewed.
    • concurrent read
    • concurrent write
    • concurrent read and write
  • added comments explaining the why and the intent of the code wherever it would not be obvious.
  • added unit tests or modified existing tests to cover new code paths.

Key changed/added classes (or packages if there are too many classes) in this PR
  • ConsensusSubscriptionPrefetchingQueueMetrics
  • ConsensusPrefetchingQueue
  • SubscriptionReceiverV1
  • Metric

@jt2594838
jt2594838 merged commit 59b019a into apache:master Jul 22, 2026
41 checks passed
@jt2594838
jt2594838 deleted the fix/consensus-subscription-metrics-region branch July 22, 2026 10:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants