Add Gossip Loop metrics #26195
Merged
Add Gossip Loop metrics #26195
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- fix nav link - add bounty split policy for duplicate reports
…a-labs#25688) * Define shuffle to prep using same shuffle for multiple slices * Determine transaction indexes and plumb to execute_batch * Pair transaction_index with transaction in TransactionStatusService * Add new ReplicaTransactionInfoVersion * Plumb transaction_indexes through BankingStage * Prepare BankingStage to receive transaction indexes from PohRecorder * Determine transaction indexes in PohRecorder; add field to WorkingBank * Add PohRecorder::record unit test * Only pass starting_transaction_index around PohRecorder * Add helper structs to simplify test DashMap * Pass entry and starting-index into process_entries_with_callback together * Add tx-index checks to test_rebatch_transactions * Revert shuffle definition and use zip/unzip * Only zip/unzip if randomize * Add confirm_slot_entries test * Review nits * Add type alias to make sender docs more clear
finish filling out the table....
…a into ftr-gossip-loop-metrics
carllin
reviewed
Jun 24, 2022
gossip/src/cluster_info_metrics.rs
Outdated
| @@ -113,6 +113,10 @@ pub struct GossipStats { | |||
| pub(crate) gossip_pull_request_verify_fail: Counter, | |||
| pub(crate) gossip_pull_response_verify_fail: Counter, | |||
| pub(crate) gossip_push_msg_verify_fail: Counter, | |||
| pub(crate) gossip_transmit_loop_time: Counter, | |||
| pub(crate) gossip_transmit_loop_itrs_since_last_report: Counter, | |||
There was a problem hiding this comment.
nit: itrs -> iterations we usually spell out all words without abbreviations
behzadnouri
previously approved these changes
Jun 24, 2022
gossip/src/cluster_info_metrics.rs
Outdated
Comment on lines
245
to
249
| ( | ||
| "process_gossip_packets_itrs_since_last_report", | ||
| stats.process_gossip_packets_itrs_since_last_report.clear(), | ||
| i64 | ||
| ), |
There was a problem hiding this comment.
for easier queries, can you please report this in the same table as the other ones?
gossip/src/cluster_info_metrics.rs
Outdated
| @@ -138,6 +142,7 @@ pub struct GossipStats { | |||
| pub(crate) packets_sent_pull_responses_count: Counter, | |||
| pub(crate) packets_sent_push_messages_count: Counter, | |||
| pub(crate) process_gossip_packets_time: Counter, | |||
| pub(crate) process_gossip_packets_itrs_since_last_report: Counter, | |||
There was a problem hiding this comment.
can you please keep the fields in this struct alphabetically sorted ?
that makes it easier to resolve merge conflicts or skim through the list of metrics.
mergify
bot
dismissed
behzadnouri’s stale review
Pull request has been modified.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
There is little insight into how well the gossip protocol is running. We need gossip timing metrics for:
run_gossip()run_listen()process_packets()Summary of Changes
Added two timers the measure the loop time of the
run_gossip()andrun_listen()methods respectively.gossip_transmit_loop_timegossip_listen_loop_timeNote a timer metric called:
process_gossip_packets_timealready exists forprocess_packets().Added three loop counters that count the total number of loop iterations since the last metric report for all the of the above loops/
gossip_transmit_loop_itrs_since_last_reportgossip_listen_loop_itrs_since_last_reportprocess_gossip_packets_itrs_since_last_reportFixes #