Auto negotiation for ZMQ_CURVE_SERVER option

[rustedsword]

Currently, to use CureZMQ i have to designate one socket as 'curve server' and another one as 'curve client', But, for example, when using ROUTER-ROUTER connections if i set ZMQ_CURVE_SERVER for one router socket, then this automatically means that all other router sockets which i'm going to connect must be 'curve clients', but then these router sockets can't connect to each other, because they are 'curve clients'.

Of course this problem can be solved by using dealer sockets as clients, and router as server for each peer, just like in zyre, but router socket with all it's features currently implemented and planned is already good p2p solution and managing hundreds of dealer sockets is not so fun.

So, as i understand zeromq, to make CurveZMQ work for random ROUTER-ROUTER connections there should be a mechanism for auto negotiation who is 'curve server' and who is 'curve client' for current connection. One downside of this - all router sockets must use same public/secret key pair.

In specification for ZMTP 3.0 there is field 'as_server' which should be set to 1 if remote side is acting as server for authentication, currently libzmq always sets this to 0. If we set this field to 1 then remote side will know that incoming connection is from 'server side'. Maybe if this is inbound connection and if we are acting as_server too then switch to client mode and send as_server = 0 to this peer?

Any thoughts on this?

[bluca]

Yes this is something that has always annoyed me - that part of the spec is ignored. The problem is backward compatibility - any solution must not break backward compatibility with both older versions of libzmq and other full implementations of ZMTP (there's a few in Erlang, Java, C# etc) :-(

The other issue is that for it to be sensible it's the binding side (ie: the one being contacted, not the one doing the connection) that should be the server-ish.

Did you already code a solultion? If you have one please do send a PR, I'd be really interested in fixing this problem

[rustedsword]

Yes this is something that has always annoyed me - that part of the spec is ignored. The problem is backward compatibility - any solution must not break backward compatibility with both older versions of libzmq and other full implementations of ZMTP (there's a few in Erlang, Java, C# etc) :-(

Hmm.. as_server is always 0 is because of backward compatibility? =/

Did you already code a solultion? If you have one please do send a PR, I'd be really interested in fixing this problem

No, not yet. I'm new to zeromq, so i just wanted to hear some opinion from developers first before writing code.

I thought about implementing new option: ZMQ_CURVE_SERVER_AUTO. When this option is set to 1 and if inbound connection with as_server = 1 in greeting received then socket will act as client for this connection and as_server = 0 will be sent to connecting client. This way no changes in ZMTP specification required. Only fixing as_server field in libzmq greeting and adding additional logic for this new option.

But again, i am not zeromq expert, so maybe there is something wrong with this solution.

[bluca]

Unfortunately adding a new option doesn't help - if the option is on, then compatibility is broken. That's not good enough I'm afraid.

[fbacchella]

A new socket option is the good solution, or a least a part of it. In controlled or test environments, one know which version it talk too and can dedicate server socket for old or new kind connection.
At least a comment should be added in code, to avoid having this issue popping again and again.

[fbacchella]

The option could set two other values: 2 for mandatory client, 4 for mandatory server.

[justusranvier]

The libzmq documentation seems to imply that a socket can change roles between client and server without affecting already-established connections, but it's a bit vague about how that actually works.

Suppose I configure socket 1 as a curve server then bind to an endpoint. If socket 2, which is configured as a curve client, connects to my endpoint then everything works.

Now suppose I configure socket 1 as a client and then connect to socket 3 which is configured as a server. I've never actually tested this but my reading of the documentation suggests it will succeed without affecting the existing connection to socket 2.

What isn't clear is what happens now if socket 4 attempts to connect to socket 1 as a client? Will the connection fail because socket 1 is now in client mode, or will it succeed because it was in server mode when the endpoint was bound?

If I want this configuration to keep working do I have to switch socket 1 to client mode, then connect to socket 3, then immediately switch socket 1 back to server mode and just hope nobody attempted to connect during that interval?

[fbacchella]

The byte I’m taking about is only used during the handshake of the OS socket connection, and use the setup of the ZMQ socket at this time. It’s used by the security mechanism that are usually not symmetric.
The scenario you describe make no sense. ZMQ socket can reconnect the OS socket at any time, without any notification, that the very purpose of the library. Tweaking with those kind of settings, or other low level settings have no chance to works in an efficient way. Locking them after the bind/connect might be a good solution, but that’s a different story.