-
Notifications
You must be signed in to change notification settings - Fork 564
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Allow to broadcast network messages in parallel #1409
Changes from 11 commits
a707610
fe6fb62
f1256a0
05b27d3
605f947
3822198
c900596
e359f96
1c4dd2e
c1e25f6
ee05f60
5eb265b
0324a41
969c036
00f2fdf
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -20,7 +20,10 @@ use super::*; | |
|
||
use always_assert::never; | ||
use bytes::Bytes; | ||
use futures::stream::BoxStream; | ||
use futures::{ | ||
future::BoxFuture, | ||
stream::{BoxStream, FuturesUnordered, StreamExt}, | ||
}; | ||
use parity_scale_codec::{Decode, DecodeAll}; | ||
|
||
use sc_network::Event as NetworkEvent; | ||
|
@@ -1038,21 +1041,58 @@ fn dispatch_collation_event_to_all_unbounded( | |
} | ||
} | ||
|
||
fn try_send_validation_event<E>( | ||
event: E, | ||
sender: &mut (impl overseer::NetworkBridgeRxSenderTrait + overseer::SubsystemSender<E>), | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nit: add the trait bound to the |
||
delayed_queue: &FuturesUnordered<BoxFuture<'static, ()>>, | ||
) where | ||
E: Send + 'static, | ||
{ | ||
match sender.try_send_message(event) { | ||
Ok(()) => {}, | ||
Err(overseer::TrySendError::Full(event)) => { | ||
let mut sender = sender.clone(); | ||
delayed_queue.push(Box::pin(async move { | ||
sender.send_message(event).await; | ||
})); | ||
}, | ||
Err(overseer::TrySendError::Closed(_)) => { | ||
panic!( | ||
"NetworkBridgeRxSender is closed when trying to send event of type: {}", | ||
std::any::type_name::<E>() | ||
); | ||
}, | ||
} | ||
} | ||
|
||
async fn dispatch_validation_events_to_all<I>( | ||
events: I, | ||
sender: &mut impl overseer::NetworkBridgeRxSenderTrait, | ||
) where | ||
I: IntoIterator<Item = NetworkBridgeEvent<net_protocol::VersionedValidationProtocol>>, | ||
I::IntoIter: Send, | ||
{ | ||
let delayed_messages: FuturesUnordered<BoxFuture<'static, ()>> = FuturesUnordered::new(); | ||
|
||
// Fast path for sending events to subsystems, if any subsystem's queue is full, we hold | ||
// the slow path future in the `delayed_messages` queue. | ||
for event in events { | ||
sender | ||
.send_messages(event.focus().map(StatementDistributionMessage::from)) | ||
.await; | ||
sender.send_messages(event.focus().map(BitfieldDistributionMessage::from)).await; | ||
sender.send_messages(event.focus().map(ApprovalDistributionMessage::from)).await; | ||
sender.send_messages(event.focus().map(GossipSupportMessage::from)).await; | ||
if let Ok(msg) = event.focus().map(StatementDistributionMessage::from) { | ||
try_send_validation_event(msg, sender, &delayed_messages); | ||
} | ||
if let Ok(msg) = event.focus().map(BitfieldDistributionMessage::from) { | ||
try_send_validation_event(msg, sender, &delayed_messages); | ||
} | ||
if let Ok(msg) = event.focus().map(ApprovalDistributionMessage::from) { | ||
try_send_validation_event(msg, sender, &delayed_messages); | ||
} | ||
if let Ok(msg) = event.focus().map(GossipSupportMessage::from) { | ||
try_send_validation_event(msg, sender, &delayed_messages); | ||
} | ||
} | ||
|
||
// Here we wait for all the delayed messages to be sent. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I suggest to add a metric that tells us how much we wait here. This will be helpful to measure the level of back pressure. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think the best metric type here will be a Histogram, on the other hand it will be quite expensive in terms of the metrics space. WDYT about adding a histogram with low amount of buckets (like There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'd say it would be sufficient to just have a counter that measure how much time is spent there. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. But we still need a histogram for time I suppose. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. we could also add a metric for the size of the |
||
let _: Vec<()> = delayed_messages.collect().await; | ||
} | ||
|
||
async fn dispatch_collation_events_to_all<I>( | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I find the name a bit misleading. The
try
suggest that we don't wait when full - which is true, but we also don't give up. Maybe send_or_queue_validation_event?