Skip to content

Commit

Permalink
KAFKA-15293 Added documentation for tiered storage metrics (apache#14331
Browse files Browse the repository at this point in the history
)

Reviewers: Divij Vaidya <diviv@amazon.com>, Satish Duggana <satishd@apache.org>
  • Loading branch information
abhijeetk88 authored and mjsax committed Nov 22, 2023
1 parent bd7dd29 commit 6ae0c9b
Show file tree
Hide file tree
Showing 2 changed files with 58 additions and 0 deletions.
57 changes: 57 additions & 0 deletions docs/ops.html
Expand Up @@ -1841,6 +1841,63 @@ <h4 class="anchor-heading"><a id="remote_jmx" class="anchor-link"></a><a href="#
</tr>
</tbody></table>

<h4 class="anchor-heading"><a id="tiered_storage_monitoring" class="anchor-link"></a><a href="#tiered_storage_monitoring">Tiered Storage Monitoring</a></h4>
The following set of metrics are available for monitoring of the tiered storage feature:<br/><br/>
<table class="data-table">
<tbody>
<tr>
<th>Metric/Attribute name</th>
<th>Description</th>
<th>Mbean name</th>
</tr>
<tr>
<td>Remote Fetch Bytes Per Sec</td>
<td>Rate of bytes read from remote storage per topic. Omitting 'topic=(...)' will yield the all-topic rate</td>
<td>kafka.server:type=BrokerTopicMetrics,name=RemoteFetchBytesPerSec,topic=([-.\w]+)</td>
</tr>
<tr>
<td>Remote Fetch Requests Per Sec</td>
<td>Rate of read requests from remote storage per topic. Omitting 'topic=(...)' will yield the all-topic rate</td>
<td>kafka.server:type=BrokerTopicMetrics,name=RemoteFetchRequestsPerSec,topic=([-.\w]+)</td>
</tr>
<tr>
<td>Remote Fetch Errors Per Sec</td>
<td>Rate of read errors from remote storage per topic. Omitting 'topic=(...)' will yield the all-topic rate</td>
<td>kafka.server:type=BrokerTopicMetrics,name=RemoteFetchErrorsPerSec,topic=([-.\w]+)</td>
</tr>
<tr>
<td>Remote Copy Bytes Per Sec</td>
<td>Rate of bytes copied to remote storage per topic. Omitting 'topic=(...)' will yield the all-topic rate</td>
<td>kafka.server:type=BrokerTopicMetrics,name=RemoteCopyBytesPerSec,topic=([-.\w]+)</td>
</tr>
<tr>
<td>Remote Copy Requests Per Sec</td>
<td>Rate of write requests to remote storage per topic. Omitting 'topic=(...)' will yield the all-topic rate</td>
<td>kafka.server:type=BrokerTopicMetrics,name=RemoteCopyRequestsPerSec,topic=([-.\w]+)</td>
</tr>
<tr>
<td>Remote Copy Errors Per Sec</td>
<td>Rate of write errors from remote storage per topic. Omitting 'topic=(...)' will yield the all-topic rate</td>
<td>kafka.server:type=BrokerTopicMetrics,name=RemoteCopyErrorsPerSec,topic=([-.\w]+)</td>
</tr>
<tr>
<td>RemoteLogReader Task Queue Size</td>
<td>Size of the queue holding remote storage read tasks</td>
<td>org.apache.kafka.storage.internals.log:type=RemoteStorageThreadPool,name=RemoteLogReaderTaskQueueSize</td>
</tr>
<tr>
<td>RemoteLogReader Avg Idle Percent</td>
<td>Average idle percent of thread pool for processing remote storage read tasks</td>
<td>org.apache.kafka.storage.internals.log:type=RemoteStorageThreadPool,name=RemoteLogReaderAvgIdlePercent</td>
</tr>
<tr>
<td>RemoteLogManager Tasks Avg Idle Percent</td>
<td>Average idle percent of thread pool for copying data to remote storage</td>
<td>kafka.log.remote:type=RemoteLogManager,name=RemoteLogManagerTasksAvgIdlePercent</td>
</tr>
</tbody>
</table>

<h4 class="anchor-heading"><a id="kraft_monitoring" class="anchor-link"></a><a href="#kraft_monitoring">KRaft Monitoring Metrics</a></h4>
The set of metrics that allow monitoring of the KRaft quorum and the metadata log.<br>
Note that some exposed metrics depend on the role of the node as defined by <code>process.roles</code>
Expand Down
1 change: 1 addition & 0 deletions docs/toc.html
Expand Up @@ -141,6 +141,7 @@
<li><a href="#monitoring">6.8 Monitoring</a>
<ul>
<li><a href="#remote_jmx">Security Considerations for Remote Monitoring using JMX</a></li>
<li><a href="#tiered_storage_monitoring">Tiered Storage Monitoring</a></li>
<li><a href="#kraft_monitoring">KRaft Monitoring</a></li>
<li><a href="#selector_monitoring">Selector Monitoring</a></li>
<li><a href="#common_node_monitoring">Common Node Monitoring</a></li>
Expand Down

0 comments on commit 6ae0c9b

Please sign in to comment.