Implement support for sending RPC to known but disconnected peer

[aschran]

This should have the same interface as any regular RPC send, just treating the time it takes for anemo to connect to the peer the same way as any other kind of latency.

Otherwise every client that needs reliable send (or not really reliable, but more like, "keep trying until explicitly told to give up") will have to reimplement essentially the same logic on the application side.

[aschran]

I think it's ok to make this non-default and require users to opt in on a specific call by, for example, retrieving a different kind of Peer than normal (a RetryingPeer or something).

[aschran]

One more thought, while one could imagine implementing this at the application level by adding retries, that is worse than the network stack doing this because the application doesn't necessarily have visibility into the state of underlying connections. So maybe I'm retrying every 1s but the peer connects after 100ms, and then I'm just sitting around waiting for 900ms until the next retry. Whereas if this is queued in the network stack then as soon as the connection happens, the request can immediately send.

This is indeed a deficiency of the current P2pNetwork::send implementation in narwhal today.

[bmwill]

Closed by MystenLabs/sui#5277