Skip to content

HDDS-8746. Add metrics to ReplicationSupervisor for task count and max streams#4818

Merged
sodonnel merged 1 commit intoapache:masterfrom
sodonnel:HDDS-8746
Jun 2, 2023
Merged

HDDS-8746. Add metrics to ReplicationSupervisor for task count and max streams#4818
sodonnel merged 1 commit intoapache:masterfrom
sodonnel:HDDS-8746

Conversation

@sodonnel
Copy link
Contributor

@sodonnel sodonnel commented Jun 1, 2023

What changes were proposed in this pull request?

To give added visibility into the Replication Queue on a datanode, this Jira adds some extra metrics to give visibility into the count of "normal priority" Replication and Reconstruction tasks, along with the current maxStreams setting, which is automatically adjusted when the node moves into decommission.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-8746

How was this patch tested?

This was tested manually in Docker compose by sampling the JMX output. Sample output:

    "name" : "Hadoop:service=HddsDatanode,name=ReplicationSupervisorMetrics",
    "modelerType" : "ReplicationSupervisorMetrics",
    "tag.Hostname" : "b297306fcf0f",
    "numInFlightReplications" : 5,
    "numQueuedReplications" : 4,
    "numRequestedReplications" : 1,
    "numTimeoutReplications" : 0,
    "numSkippedReplications" : 0,
    "maxReplicationStreams" : 1, ** newly added
    "numInflightECReconstructionCoordinatorTask" : 5 ** newly added


    "name" : "Hadoop:service=HddsDatanode,name=ReplicationSupervisorMetrics",
    "modelerType" : "ReplicationSupervisorMetrics",
    "tag.Hostname" : "b297306fcf0f",
    "numInFlightReplications" : 1,
    "numQueuedReplications" : 0,
    "numRequestedReplications" : 6,
    "numTimeoutReplications" : 0,
    "numSkippedReplications" : 0,
    "maxReplicationStreams" : 1,      ** newly added
    "numInflightReplicationTask" : 1,  ** newly added
    "numInflightECReconstructionCoordinatorTask" : 0 ** newly added

Copy link
Contributor

@adoroszlai adoroszlai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @sodonnel for the patch, LGTM.

@sodonnel sodonnel merged commit e7b816d into apache:master Jun 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants