Adding Dispute Participation Metrics #6838

BradleyOlson64 · 2023-03-07T23:17:19Z

Per issue #4393

We should have metrics on the following:

How long does the actual participation task take
How long does it take from queuing a participation to the participation task finishing
Participation queue sizes (best effort & priority)

…brad-issue-4393

node/core/dispute-coordinator/src/metrics.rs

mrcnski · 2023-03-09T10:05:38Z

node/core/dispute-coordinator/src/participation/queues/mod.rs

+	/// Timer handle for each participation request. Stored to measure full request
+	/// completion time. Optimally these would have been stored in the participation
+	/// request itself, but HistogramTimer doesn't implement the Clone trait.
+	request_timers: BTreeMap<CandidateComparator, Option<prometheus::HistogramTimer>>,


HistogramTimer doesn't implement the Clone trait.

I think you can put it behind a smart pointer like Rc. Might simplify the code. 🙂

Rc doesn't work because we have multithreaded tokio runtime, we would need Arc.

Placed the timer in ParticipationRequest. It does indeed simplify the changes 👍 Please check the refactor when you have a moment!

node/core/dispute-coordinator/src/metrics.rs

sandreim · 2023-03-09T10:28:35Z

node/core/dispute-coordinator/src/participation/queues/mod.rs

+	/// Timer handle for each participation request. Stored to measure full request
+	/// completion time. Optimally these would have been stored in the participation
+	/// request itself, but HistogramTimer doesn't implement the Clone trait.
+	request_timers: BTreeMap<CandidateComparator, Option<prometheus::HistogramTimer>>,


Rc doesn't work because we have multithreaded tokio runtime, we would need Arc.

mrcnski · 2023-03-10T12:52:23Z

node/core/dispute-coordinator/src/metrics.rs

@@ -221,12 +221,12 @@ impl metrics::Metrics for Metrics {
 				)?,
 				registry,
 			)?,
-			priority_queue_size: prometheus::register(
+			participation_priority_queue_size: prometheus::register(
 				prometheus::Gauge::new("polkadot_parachain_dispute_priority_queue_size", 


nit: Should the metric names be updated to match the new variable names? e.g. "polkadot_parachain_dispute_participation_priority_queue_size"

mrcnski · 2023-03-10T12:58:31Z

node/core/dispute-coordinator/src/initialized.rs

+			let request_timer = Arc::new(self.metrics.time_participation_pipeline());
 			let r = self
 				.participation
 				.queue_participation(
 					ctx,
 					priority,
-					ParticipationRequest::new(new_state.candidate_receipt().clone(), session),
+					ParticipationRequest::new(new_state.candidate_receipt().clone(), session, request_timer),


My first reaction was maybe request_timer can be initialized in ParticipationRequest::new. But after looking at it, doing it like this instead of hiding it inside ParticipationRequest::new makes it more explicit that we are starting a timer here. 👍

mrcnski · 2023-03-10T13:09:22Z

node/core/dispute-coordinator/src/participation/queues/mod.rs

+		if  &self.candidate_receipt == other.candidate_receipt() &&
+			&self.candidate_hash == other.candidate_hash() &&
+			self.session == other.session()


I would destructure self into its fields, so that if a field is ever added in the future we don't forget to add it to the comparison here. It would look something like this:

let ParticipationRequest { candidate_receipt, candidate_hash, session, _request_timer } = self; /* do the comparison */

Credit to @eskimor for teaching me this!

nit: Also, instead of if cond { return true; } false, you can simplify it to just cond. :) I think clippy should warn about this.

Credit to @eskimor for teaching me this!

+1 :)

mrcnski · 2023-03-10T13:23:41Z

node/core/dispute-coordinator/src/participation/queues/mod.rs

+	// eq. This helper checks whether all other fields are equal,
+	// which is sufficient.
+	#[cfg(test)]
+	pub fn functionally_equal(&self, other: ParticipationRequest) -> bool {


nit: I had an internal struggle about this, but I think we should just impl eq on this type. I believe that already implies functional equality. I read the docs for PartialEq, and there is no restriction mentioned about having to include every field in the equality comparison.

Interesting! I hadn't thought of it that way. But it does make sense. I shall make changes.

mrcnski

Looks good! Left some nits, but nothing blocking.

node/core/dispute-coordinator/src/participation/mod.rs

tdimitrov · 2023-03-10T13:50:57Z

node/core/dispute-coordinator/src/participation/mod.rs

@@ -235,7 +241,8 @@ impl Participation {
 		recent_head: Hash,
 	) -> FatalResult<()> {
 		while self.running_participations.len() < MAX_PARALLEL_PARTICIPATIONS {
-			if let Some(req) = self.queue.dequeue() {
+			let maybe_req = self.queue.dequeue();


nit: Why we need this variable?

Good catch! That was an artifact from a discarded implementation.

tdimitrov · 2023-03-10T13:55:34Z

node/core/dispute-coordinator/src/participation/queues/mod.rs

+	// which is sufficient.
+	#[cfg(test)]
+	pub fn functionally_equal(&self, other: ParticipationRequest) -> bool {
+		if &self.candidate_receipt == other.candidate_receipt() &&


I'd implement this by destructuring (unless it's not possible due to something I'm not seeing right now). This way we'll get a compilation error if a new field is added to the struct.

Also can't we implement std::cmp::PartialEq::eq for ParticipationRequest so that we have got a bit nicer looking syntax?

node/core/dispute-coordinator/src/participation/queues/mod.rs

tdimitrov · 2023-03-10T14:17:34Z

Good job! Just some small remarks from me.

tdimitrov · 2023-03-10T17:53:55Z

I just saw that @mrcnski has already left comments similar to mine. Sorry for the double posting :/

…brad-issue-4393

This reverts commit 02e5608.

BradleyOlson64 · 2023-03-11T00:38:06Z

bot merge

node/core/dispute-coordinator/src/lib.rs

node/core/dispute-coordinator/src/participation/mod.rs

* master: (50 commits) Issue 4393: Correcting Unnecessary Use of Arc (#6882) Companion for #13287 (#6655) timestamp ci job logs (#6890) Release parachain host API v4 (#6885) polkadot companion: #13128 (Pools commission) (#6264) companion for #13555 (#6842) Bump libgit2-sys from 0.14.1+1.5.0 to 0.14.2+1.5.1 (#6600) Bump bumpalo from 3.8.0 to 3.12.0 (#6599) Bump git2 from 0.16.0 to 0.16.1 (#6601) Council as SpendOrigin (#6877) PVF: Document that preparation cannot lead to disputes (#6873) Sync versions with current release (0.9.39) (#6875) Companion for paritytech/substrate#13592 (#6869) Update orchestra to the recent version (#6854) Delete unused Cargo.lock (#6870) Remove use of Store trait (#6835) Companion for Substrate #13564 (#6845) Adding Dispute Participation Metrics (#6838) Update `substrate` to 48e7cb1 (#6851) Move PVF timeouts to executor environment parameters (#6823) ...

…slashing-client * ao-past-session-slashing-runtime: (23 commits) Issue 4393: Correcting Unnecessary Use of Arc (#6882) Companion for #13287 (#6655) timestamp ci job logs (#6890) Release parachain host API v4 (#6885) polkadot companion: #13128 (Pools commission) (#6264) companion for #13555 (#6842) Bump libgit2-sys from 0.14.1+1.5.0 to 0.14.2+1.5.1 (#6600) Bump bumpalo from 3.8.0 to 3.12.0 (#6599) Bump git2 from 0.16.0 to 0.16.1 (#6601) Council as SpendOrigin (#6877) PVF: Document that preparation cannot lead to disputes (#6873) Sync versions with current release (0.9.39) (#6875) Companion for paritytech/substrate#13592 (#6869) Update orchestra to the recent version (#6854) Delete unused Cargo.lock (#6870) Remove use of Store trait (#6835) Companion for Substrate #13564 (#6845) Adding Dispute Participation Metrics (#6838) Update `substrate` to 48e7cb1 (#6851) Move PVF timeouts to executor environment parameters (#6823) ...

BradleyOlson64 added 4 commits March 6, 2023 17:56

Added participation and queue sizes metrics

107f971

First draft of all metric code

a8186fc

Tests pass

b6cb2c7

Merge branch 'master' of https://github.com/paritytech/polkadot into …

faa7eb5

…brad-issue-4393

BradleyOlson64 requested review from tdimitrov and eskimor March 7, 2023 23:17

BradleyOlson64 self-assigned this Mar 7, 2023

BradleyOlson64 added A0-please_review Pull request needs code review. B0-silent Changes should not be mentioned in any release notes C1-low PR touches the given topic and has a low impact on builders. labels Mar 7, 2023

BradleyOlson64 linked an issue Mar 7, 2023 that may be closed by this pull request

dispute-coordinator: Add metrics on participation performance #4393

Closed

BradleyOlson64 added 2 commits March 8, 2023 11:43

Changed Metrics to field on participation + queues

bc7049e

fmt

7b0bfc0

mrcnski approved these changes Mar 9, 2023

View reviewed changes

sandreim reviewed Mar 9, 2023

View reviewed changes

BradleyOlson64 added 3 commits March 9, 2023 09:00

Improving naming

56e7723

Refactor, placing timer in ParticipationRequest

4185d4d

fmt

20ceda9

BradleyOlson64 removed a link to an issue Mar 9, 2023

dispute-coordinator: Add metrics on participation performance #4393

Closed

sandreim approved these changes Mar 10, 2023

View reviewed changes

mrcnski reviewed Mar 10, 2023

View reviewed changes

mrcnski approved these changes Mar 10, 2023

View reviewed changes

tdimitrov reviewed Mar 10, 2023

View reviewed changes

node/core/dispute-coordinator/src/participation/mod.rs Show resolved Hide resolved

tdimitrov reviewed Mar 10, 2023

View reviewed changes

Final cleanup

02e5608

Merge branch 'master' of https://github.com/paritytech/polkadot into …

6444e6d

…brad-issue-4393

tdimitrov approved these changes Mar 10, 2023

View reviewed changes

BradleyOlson64 added 4 commits March 10, 2023 12:48

Revert "Final cleanup"

008063f

This reverts commit 02e5608.

Changing metric names

207bb2a

Implementing Eq only for unit tests

3fcb639

fmt

ef611aa

BradleyOlson64 removed the request for review from eskimor March 11, 2023 00:00

paritytech-processbot bot merged commit a8e6828 into master Mar 11, 2023

paritytech-processbot bot deleted the brad-issue-4393 branch March 11, 2023 00:38

BradleyOlson64 mentioned this pull request Mar 14, 2023

Add Guide For Network Scale Testing (Versi) paritytech/polkadot-sdk#691

Open

eskimor reviewed Mar 14, 2023

View reviewed changes

node/core/dispute-coordinator/src/lib.rs Show resolved Hide resolved

node/core/dispute-coordinator/src/participation/mod.rs Show resolved Hide resolved

BradleyOlson64 mentioned this pull request Mar 14, 2023

Issue 4393: Correcting Unnecessary Use of Arc #6882

Merged

BradleyOlson64 mentioned this pull request Mar 16, 2023

Handling timers for repeat dispute participation requests #6901

Merged

kiltbot mentioned this pull request Apr 3, 2023

3️⃣ [AUTOMATIC] Update Polkadot dependencies from 0.9.39 to 0.9.40 KILTprotocol/kilt-node#496

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adding Dispute Participation Metrics #6838

Adding Dispute Participation Metrics #6838

BradleyOlson64 commented Mar 7, 2023

mrcnski Mar 9, 2023

sandreim Mar 9, 2023

BradleyOlson64 Mar 9, 2023

sandreim Mar 9, 2023

mrcnski Mar 10, 2023

mrcnski Mar 10, 2023

mrcnski Mar 10, 2023 •

edited

Loading

mrcnski Mar 10, 2023

tdimitrov Mar 10, 2023

mrcnski Mar 10, 2023 •

edited

Loading

BradleyOlson64 Mar 10, 2023

mrcnski left a comment

tdimitrov Mar 10, 2023

BradleyOlson64 Mar 10, 2023

tdimitrov Mar 10, 2023

tdimitrov Mar 10, 2023

tdimitrov commented Mar 10, 2023

tdimitrov commented Mar 10, 2023 •

edited

Loading

BradleyOlson64 commented Mar 11, 2023

Adding Dispute Participation Metrics #6838

Adding Dispute Participation Metrics #6838

Conversation

BradleyOlson64 commented Mar 7, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrcnski Mar 10, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrcnski Mar 10, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mrcnski left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

tdimitrov commented Mar 10, 2023

tdimitrov commented Mar 10, 2023 • edited Loading

BradleyOlson64 commented Mar 11, 2023

mrcnski Mar 10, 2023 •

edited

Loading

mrcnski Mar 10, 2023 •

edited

Loading

tdimitrov commented Mar 10, 2023 •

edited

Loading