Beacon Refreshing Peers

[smalls12]

I've been testing with pyre( https://github.com/zeromq/pyre ) and aspyre( https://github.com/smalls12/aspyre ) for its compatibility with zyre, which is the de-facto implementation of the ZRE protocol, because ideally all three would be compatible with each other.

I noticed that pyre and aspyre treat beacons as a means to refresh a peer so as to not leave the node to think that peer is being evasive.

zyre however does not do this.

I checked inside zyre_node.c and confirmed that zyre_node_require_peer will not refresh an already discovered peer.

Any reason this could not be added?

[smalls12]

Confirmed from RFC

The PING Command

A node SHOULD send a PING command to any peer that it has not received a UDP beacon from within a certain time (typically five seconds). Note that UDP traffic may be dropped on a network that is heavily saturated. If a node receives no reply to a PING command, and no other traffic from a peer within a somewhat longer time (typically 30 seconds), it SHOULD treat that peer as dead.

Note that PING commands SHOULD be used only in targeted cases where a peer is otherwise silent. Otherwise, the cost of PING commands will rise exponentially with the number of peers connected, and can degrade network performance.

[brettviren]

Maybe related: #643

[smalls12]

Could be? Definitely in the same vein.

My initial thoughts were along the lines of networks with large amounts of nodes ( say 100 ) where all they are doing is waiting for messages to come in; so idling.

Every 5 seconds, every node will think the other 99 are being EVASIVE and will ping each peer, so the network gets ~10k PINGs followed by ~10k PING_OKs; and of course 99 EVASIVE messages being propagated to the user of each node.

Seems excessive.

[smalls12]

One comment I noticed was whether a beacon would necessarily indicate that a peer is available, thus prompting the need to maybe choose to force a ping.

it was mentioned in #564 (comment) that pinging when a node becomes EVASIVE would be useful when there are multiple network interfaces. In case the connection on one of the interfaces goes down, beacon signal alone is not enough to know whether a node can communicate. At least that's how I understood it. I have not had this use case, so I'm not sure if it would actually be useful. So far, we just ignore the EVASIVE event and only listen to EXIT to know if a node has left the network.

An alternative here though would be to bind to the correct interface for the beacon to use though I would think.

[smalls12]

Maybe getting really fancy, the DEALER socket could have a socket monitor to see any potential underlying socket errors.

But I suspect that would still be impacted by inherent lossy networks and no errors would not be immediately detected.

[smalls12]

Another thought on multi-NIC setups.
As it is now, we have a single beacon for all interfaces.
Could switch to allow for multiple beacons for a one to one mapping of beacons to interfaces.

I'm just rambling.

[sphaero]

Pyre and Zyre are not very compatible anymore from what I recall. Have you also considered testing the python binding of zyre?

[smalls12]

From my testing they do work together but there are some things implemented in zyre that are not in pyre.

These include support for the new draft https://rfc.zeromq.org/spec/43/, namely curve support.
Elections are in zyre but are not part of the RFC.
EVASIVE and SILENT are in zyre but are not in the RFC.

My goal is to bring them in as part of my other rework of pyre for asyncio.
https://github.com/smalls12/aspyre

[stale]

This issue has been automatically marked as stale because it has not had recent activity for 90 days. It will be closed if no further activity occurs within 21 days. Thank you for your contributions.