Improve the throughput of Actors using ZeroMQ and the Request/Reply pattern.

[alexandreagular]

Following this discussion https://groups.google.com/forum/#!topic/akka-user/lGBb8GTouV0

I propose a new implementation of the ConcurrentSocketActor in akka-zeromq.

This new version does not rely on polling at all and has faster throughput.

I also made a sample application (akka-sample-zeromq-reqrep) for testing purpose.

[viktorklang]

Hey! Have all contributors to the code signed the CLA?

typesafe.com/contribute/cla

[alexandreagular]

I haven't done that yet. I need to check it with my manager, which won't be until monday.

[viktorklang]

Great, let me know when you've signed or if you cannot sign. Without CLA signature I cannot do anything with this PR but to close it.

[alexandreagular]

Hi,

Just committed a new version and added some more sample code. the CLA signature should come soon.

[jboner]

Great. Let us know when you have it signed.

[jacobkolind]

I am Alexandre's manager (@jacobkolind), and I have just signed the CLA. Are we good to go if he also signs the CLA?

[viktorklang]

Hi Jacob,

everyone who has contributed to the PR has to sign the CLA. However, we have the possibility of issuing a corporate CLA for the company so that all employees' contributions are covered by that, is that something that would be interesting?

[jacobkolind]

We discussed this with our CEO, but he was of the opinion that he would rather that we signed the CLA individually. The company has waived all rights to any IP that we contribute to Akka.

[johanlaidlaw]

I have also signed the CLA so now everyone contributing to this pull request have signed the CLA. Let's hope it can be used :)

[viktorklang]

Excellent. First of all, could you split it up into logical chunks? (1 PR for the performance fix(es)) and one for the docs and samples?

[viktorklang]

This seems to only work on ZMQ >= 3.0

[viktorklang]

Correction: 2.2: https://github.com/zeromq/jzmq/pull/109/files

[alexandreagular]

Hi,

I haven't checked that, thanks. What is the minimal version of ZeroMQ that we should support ?

I can make a fix for older versions, but it's not going to be as neat. I would rather keep that one for the current and newest version of ZeroMQ and make the fix specifically for oldest versions. Would that be acceptable ?

[viktorklang]

Latest stable release is 2.2,so that has to be supported.

[alexandreagular]

So should I leave it as it is ?

[viktorklang]

Please apply this change to ConcurrentSocketActorSpec so you can run the tests:

def checkZeroMQInstallation =
try {
zmq.version match {
case ZeroMQVersion(x, y, _) if x >= 3 || (x >= 2 && y >= 2) ⇒ Unit
case version ⇒ invalidZeroMQVersion(version)
}
} catch {
case e: LinkageError ⇒ zeroMQNotInstalled
}

[viktorklang]

So, it doesn't work on 2.2: https://www.assembla.com/spaces/akka/tickets/2538-0mq-integration-broken-on-2-2#/activity/ticket:2538

[alexandreagular]

Ok, I am going to check that and do the fix on checkZeroMQInstallation now.

[alexandreagular]

Hi,

So I have found the problem, it turns out that the setReceiveTimeout method is not working. The reason is that the scala-zeromq binding does not allow this setting with zeromq < 3.0.0.

As this does not match the zeromq 2.2 spec, I was thinking that the proper way to fix it would be to change the scala-zeromq binding, this file in particular: https://github.com/valotrading/zeromq-scala-binding/blob/master/src/main/java/org/zeromq/ZMQ.java
(see the setReceiveTimeout and setSendTimeout methods)

And upgrade the required version for the scala-zeromq binding in akka

Does that sound acceptable to you ?

The other solution we have is a duplicate implementation for zeromq < 3.0.0 and >= 3.0.0 which won't be as neat.

Alexandre

[viktorklang]

What happens if one uses the java-zmq-binding?

[alexandreagular]

I checked the implementation of java-zmq-binding, it seems correct, we could use it:
https://github.com/zeromq/jzmq/blob/master/src/org/zeromq/ZMQ.java
line 586 you can see the requirement is 2.2.0

[viktorklang]

Alright, so I think your proposition is OK: Fix it in the scala-zeromq-binding
When that's done we can then let users either use the scala or the java binding to their liking, they will, however, have to upgrade to atleast 0mq 2.2.

How does that sound?

[alexandreagular]

Ok that sounds good to me.

I will work on that and come back to you when I am done.

[viktorklang]

Sounds great, thanks

[alexandreagular]

Hi,

So I fixed the problem on the scala binding. I made a pull request for that:
valotrading/zeromq-scala-binding#15

I will probably fix other things I have noticed on it but that is not relevant for this pull request.

In my last commit, I changed the required version for the scala-zeromq binding to 0.0.8-SNAPSHOT, which you will have to download from my branch, compile and publish locally to make it work.

With all that (plus a small fix I had to do) the unit tests pass.

[viktorklang]

Alright. We won't be able to do anything until the artifact is published with a stable identifier (no SNAPSHOT deps allowed)
Great work improving the binding btw!

[viktorklang]

Use ActorLogging

[davojan]

Hi. This pull-request is closed and I didn't find fixes in master. Does it mean that the problem with request/reply performance is resolved any other way? Can I rely on akka-zeromq and have reasonable performance?

[patriknw]

I don't know the status, but there is another open pull request by @alexandreagular: #1285

[alexandreagular]

Yes, I closed this pull request because it was getting very old and I thought it would be simpler to get a fresh master branch and put the changes on them.

There is another one indeed on which some comments have been made that I need to work on. I will try to do that soon in my spare time.