txZMQ object responsibilities

[oubiwann]

In my last branch (#14), I kept running into conceptual issues inside txZMQ regarding what is named what, and what's responsible for what.

This is actually not surprising, given:

1. the age of the project -- young! (and the code base is pretty sweet, even considering that)
2. the fact that we're actually using zmq socket fd's and this changes so many of the semantics from the standard Twisted approach

Regardless, here are my observations:

• ZmqEndpoint takes two parameters: a connection type ("bind" or "connect") and a ZeroMQ address
• ZmqConnection does a connect or bind depending upon the connection type set in the passed ZmqEndpoint instance.
• ZmqConnection subclasses should be able to have sensible defaults for connection type (that can be overridden if necessary), thus removing one more thing from the developers' task list when using the library
• it seems that setting the connection type should happen with a connection-oriented object
• it seems that ZmqEndpoint would be better suited just focusing on the address of an endpoint
• ZmqConnection is responsible for both connecting and the work that, traditionally in twisted, is done by Protocols and Transports: namely, writing to the fd
• both client and server code is set up in ZmqConnection; I know that things are somewhat different with ZMQ, but it will be helpful to devs to create some code isolation for the two approaches while at the same time leaving things flexible enough so devs can take full advantage of ZMQ with txZMQ

Proposed solutions (branches!):

• for txZMQ twisted APIs, use Twisted terminology: rename ZmqEndpoint to ZmqAddress
• split out connection and address configuration in ZmqEndpoint, putting connection info in ZmqConnection
• split out the "transport" (writing/reading of fds) from the ZmqConnection object (not sure what exactly... need to experiment and see what fits best after tweaking the other bits)
• split ZmqConnection into ZmqServerEndpoint and ZmqClientEndpoint

I think once these solutions are in place, the code will be very logically segmented and developers will have a very clear picture of how to use txZMQ in Twisted apps.

Thoughts?

[smira]

Sorry for being too slow to respond, just got back from the trip. Please give some time to think about that...

[oubiwann]

Sure thing!

Again, whatever you want to do is cool. No pressure.

[smira]

I was looking into the changes from your pull request while trying to understand what is the best way to make ZMQ part of Twisted. txZMQ started as a quick hack to make ZeroMQ available to Twisted application.

What I was based upon:

• we don't need any kind of ZeroMQ reactor, as we can use any existing to poll on ZMQ fd
• ZMQ socket semantics is much more complex compared to TCP/UDP socket
• as ZMQ handles reconnects, there's no real requirement to put complex logic around connect/disconnect, as usually it connects only once on application startup
• particular socket type may require 'subclasses', as we don't want users to deal with raw send/recv
• some ZMQ socket types won't work well in async program, like REQ/REP, because of strict requirement to send REPly immediately after receiving REQuest (even though XREQ/XREP works fine).

I would love to see txZMQ as simple as possible for the developer.

Even though I'm using Twisted for more than 4 years, I'm still not completely in Twisted terminology/abstract classes (probably because documentation is not that complete?). I really like the direction you're heading with the changes, but I don't feel that I'm an expert. Would it make sense if I add you to this GitHub repo? I would be glad to do testing and provide review/feedback, but I feel it would be better for you to drive through this change.

[oubiwann]

Thanks for taking the time to write up such a nice response :-)

I would love to be added to the GitHub repo, and I'd enjoy working on the changes in a peer-review environment (those are always the most fulfilling ones!).

[smira]

Thanks, Duncan! Added you to the repo.