Skip to content

TQ: Implement network config replication #9317

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: tq-sprockets-3
Choose a base branch
from
+445 −62
Open

TQ: Implement network config replication #9317

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 51 additions & 20 deletions trust-quorum/src/connection_manager.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,26 +94,11 @@ pub struct ConnToMainMsg {

#[derive(Debug)]
pub enum ConnToMainMsgInner {
Accepted {
addr: SocketAddrV6,
peer_id: BaseboardId,
},
Connected {
addr: SocketAddrV6,
peer_id: BaseboardId,
},
Received {
from: BaseboardId,
msg: PeerMsg,
},
#[expect(unused)]
ReceivedNetworkConfig {
from: BaseboardId,
config: NetworkConfig,
},
Disconnected {
peer_id: BaseboardId,
},
Accepted { addr: SocketAddrV6, peer_id: BaseboardId },
Connected { addr: SocketAddrV6, peer_id: BaseboardId },
Received { from: BaseboardId, msg: PeerMsg },
ReceivedNetworkConfig { from: BaseboardId, config: NetworkConfig },
Disconnected { peer_id: BaseboardId },
}

pub struct TaskHandle {
Expand All @@ -138,6 +123,13 @@ impl TaskHandle {
pub async fn send(&self, msg: PeerMsg) {
let _ = self.tx.send(MainToConnMsg::Msg(WireMsg::Tq(msg))).await;
}

pub async fn send_network_config(&self, config: NetworkConfig) {
let _ = self
.tx
.send(MainToConnMsg::Msg(WireMsg::NetworkConfig(config)))
.await;
}
}

impl BiHashItem for TaskHandle {
Expand Down Expand Up @@ -388,6 +380,45 @@ impl ConnMgr {
}
}

// After we have updated our network config, we should send it out to all
// peers with established connections, with the exception of the peer we
// received it from if this was not a local update.
pub async fn broadcast_network_config(
&mut self,
network_config: &NetworkConfig,
excluded_peer: Option<&BaseboardId>,
) {
for h in self
.established
.iter()
.filter(|&h| Some(&h.baseboard_id) != excluded_peer)
{
info!(
self.log,
"Sending network config";
"peer_id" => %h.baseboard_id,
"generation" => network_config.generation
);
h.task_handle.send_network_config(network_config.clone()).await;
}
Comment on lines +396 to +403
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we call send_network_config instead of duplicating its contents here? Otherwise if we change one function we have to remember to change the other function too, and at some point we will forget.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We take an efficiency hit doing things that way, because now we have to do a lookup for each peer_id that we already have a handle for. It's not that big a deal, but it feels somewhat wrong to take that hit. The primary deduplication is in the underlying call to task_handle.send_network_config, whereas here the duplication that exists is just a log message.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I could pass a logger into the task_handle I suppose, and then log the message there. That would remove the duplication.

}

pub async fn send_network_config(
&mut self,
peer_id: &BaseboardId,
network_config: &NetworkConfig,
) {
if let Some(h) = self.established.get1(peer_id) {
info!(
self.log,
"Sending network config";
"peer_id" => %h.baseboard_id,
"generation" => network_config.generation
);
h.task_handle.send_network_config(network_config.clone()).await;
}
}

/// Perform any polling related operations that the connection
/// manager must perform concurrently.
pub async fn step(
Expand Down
Loading
Loading