zmq3 #1

Open
alxzh opened this Issue Mar 3, 2013 · 6 comments

Comments

Projects
None yet
3 participants
@alxzh

alxzh commented Mar 3, 2013

Hi,

Didn't know how to ask you to be sure that you will see this request, so added this issue here :)

Could you please experiment with github.com/pebbe/zmq3 instead of go-zmq in your application? Just curious to compare with go-zmq.

Thanks,
Alex

@stephenmcd

This comment has been minimized.

Show comment
Hide comment
@stephenmcd

stephenmcd Mar 3, 2013

Owner

Hey here is fine, and thanks for the interest!

Would love to try it out but I'm not sure when I would have time to. Are you interested in doing it? It would just be a matter of adding another client in pubsub/clients.go and then allowing it to be chosen via flag args when testing.

Owner

stephenmcd commented Mar 3, 2013

Hey here is fine, and thanks for the interest!

Would love to try it out but I'm not sure when I would have time to. Are you interested in doing it? It would just be a matter of adding another client in pubsub/clients.go and then allowing it to be chosen via flag args when testing.

@alxzh

This comment has been minimized.

Show comment
Hide comment
@alxzh

alxzh Mar 3, 2013

What I've got. I used redis-2.6.10 and zeromq-3.2.2

With redis:

for i in 1 2 3 4 5 6 7 8 9 10; do go run test_client3.go -num-clients=$i -redis -quiet; done
Num clients 1 median 378177 msg/sec
Num clients 2 median 210116 msg/sec
Num clients 3 median 112867 msg/sec
Num clients 4 median 81592 msg/sec
Num clients 5 median 58054 msg/sec
Num clients 6 median 47710 msg/sec
Num clients 7 median 41567 msg/sec
Num clients 8 median 35976 msg/sec
Num clients 9 median 32091 msg/sec
Num clients 10 median 29228 msg/sec

With zmq:

for i in 1 2 3 4 5 6 7 8 9 10; do go run test_client3.go -num-clients=$i -quiet; done
Num clients 1 median 246909 msg/sec
Num clients 2 median 218200 msg/sec
Num clients 3 median 157881 msg/sec
Num clients 4 median 116148 msg/sec
Num clients 5 median 86747 msg/sec
Num clients 6 median 65870 msg/sec
Num clients 7 median 50521 msg/sec
Num clients 8 median 3306 msg/sec
Num clients 9 median 2955 msg/sec
Num clients 10 median 2630 msg/sec

So result isn't far from yours. Honestly I'm a bit disappointed with zeromq but it doesn't require centralized server and for moderate number of clients works better. So dramatic drop after 7 clients with zeromq makes me think that we missed some issue in the code. Because more or less consistent behavior it drops to sharp.

alxzh commented Mar 3, 2013

What I've got. I used redis-2.6.10 and zeromq-3.2.2

With redis:

for i in 1 2 3 4 5 6 7 8 9 10; do go run test_client3.go -num-clients=$i -redis -quiet; done
Num clients 1 median 378177 msg/sec
Num clients 2 median 210116 msg/sec
Num clients 3 median 112867 msg/sec
Num clients 4 median 81592 msg/sec
Num clients 5 median 58054 msg/sec
Num clients 6 median 47710 msg/sec
Num clients 7 median 41567 msg/sec
Num clients 8 median 35976 msg/sec
Num clients 9 median 32091 msg/sec
Num clients 10 median 29228 msg/sec

With zmq:

for i in 1 2 3 4 5 6 7 8 9 10; do go run test_client3.go -num-clients=$i -quiet; done
Num clients 1 median 246909 msg/sec
Num clients 2 median 218200 msg/sec
Num clients 3 median 157881 msg/sec
Num clients 4 median 116148 msg/sec
Num clients 5 median 86747 msg/sec
Num clients 6 median 65870 msg/sec
Num clients 7 median 50521 msg/sec
Num clients 8 median 3306 msg/sec
Num clients 9 median 2955 msg/sec
Num clients 10 median 2630 msg/sec

So result isn't far from yours. Honestly I'm a bit disappointed with zeromq but it doesn't require centralized server and for moderate number of clients works better. So dramatic drop after 7 clients with zeromq makes me think that we missed some issue in the code. Because more or less consistent behavior it drops to sharp.

@stephenmcd

This comment has been minimized.

Show comment
Hide comment
@stephenmcd

stephenmcd Mar 3, 2013

Owner

Couple of points:

Firstly the "no server" case isn't really correct for zmq - we're still running a broker, just as we run Redis.

Secondly, while the drop off with increase in clients is more significant with zmq, it's similar enough with Redis that it's hard to attribute this to a difference between them - more likely the case is that the test_client.go code is using all CPU cores, even with a single client, so it's only going to go downhill from there as we introduce parallelism that now has to contend for CPU cycles. I talk about this in a couple of areas in my original article this code was written for:

http://blog.jupo.org/2013/02/23/a-tale-of-two-queues/

To be clear, we're not trying to measure how things perform with an increase in clients: we're increasing the clients in an attempt to use as much hardware as possible to push messages onto the broker. If we really wanted to test the former, we'd need to use multiple machines.

Owner

stephenmcd commented Mar 3, 2013

Couple of points:

Firstly the "no server" case isn't really correct for zmq - we're still running a broker, just as we run Redis.

Secondly, while the drop off with increase in clients is more significant with zmq, it's similar enough with Redis that it's hard to attribute this to a difference between them - more likely the case is that the test_client.go code is using all CPU cores, even with a single client, so it's only going to go downhill from there as we introduce parallelism that now has to contend for CPU cycles. I talk about this in a couple of areas in my original article this code was written for:

http://blog.jupo.org/2013/02/23/a-tale-of-two-queues/

To be clear, we're not trying to measure how things perform with an increase in clients: we're increasing the clients in an attempt to use as much hardware as possible to push messages onto the broker. If we really wanted to test the former, we'd need to use multiple machines.

@alxzh

This comment has been minimized.

Show comment
Hide comment
@alxzh

alxzh Mar 4, 2013

With different machines:

for i in 1 2 3 4 5 6 7 8 9 10; do go run test_client3.go -num-clients=$i -quiet -host=192.168.1.55 -redis; done
Num clients 1 median 43764 msg/sec
Num clients 2 median 23208 msg/sec
Num clients 3 median 16188 msg/sec
Num clients 4 median 11941 msg/sec
Num clients 5 median 9969 msg/sec
Num clients 6 median 8192 msg/sec
Num clients 7 median 7606 msg/sec
Num clients 8 median 5914 msg/sec
Num clients 9 median 5514 msg/sec
Num clients 10 median 3822 msg/sec

for i in 1 2 3 4 5 6 7 8 9 10; do go run test_client3.go -num-clients=$i -quiet -host=192.168.1.55; done
Num clients 1 median 85407 msg/sec
Num clients 2 median 44290 msg/sec
Num clients 3 median 26349 msg/sec
Num clients 4 median 20008 msg/sec
Num clients 5 median 16320 msg/sec
Num clients 6 median 14436 msg/sec
Num clients 7 median 11862 msg/sec
Num clients 8 median 7779 msg/sec
Num clients 9 median 7733 msg/sec
Num clients 10 median 8712 msg/sec

Not that bad for zmq :)

alxzh commented Mar 4, 2013

With different machines:

for i in 1 2 3 4 5 6 7 8 9 10; do go run test_client3.go -num-clients=$i -quiet -host=192.168.1.55 -redis; done
Num clients 1 median 43764 msg/sec
Num clients 2 median 23208 msg/sec
Num clients 3 median 16188 msg/sec
Num clients 4 median 11941 msg/sec
Num clients 5 median 9969 msg/sec
Num clients 6 median 8192 msg/sec
Num clients 7 median 7606 msg/sec
Num clients 8 median 5914 msg/sec
Num clients 9 median 5514 msg/sec
Num clients 10 median 3822 msg/sec

for i in 1 2 3 4 5 6 7 8 9 10; do go run test_client3.go -num-clients=$i -quiet -host=192.168.1.55; done
Num clients 1 median 85407 msg/sec
Num clients 2 median 44290 msg/sec
Num clients 3 median 26349 msg/sec
Num clients 4 median 20008 msg/sec
Num clients 5 median 16320 msg/sec
Num clients 6 median 14436 msg/sec
Num clients 7 median 11862 msg/sec
Num clients 8 median 7779 msg/sec
Num clients 9 median 7733 msg/sec
Num clients 10 median 8712 msg/sec

Not that bad for zmq :)

@stephenmcd

This comment has been minimized.

Show comment
Hide comment
@stephenmcd

stephenmcd Mar 5, 2013

Owner

Nice! I added a link to this thread from the README so people can check it out if they're interested:

ccb9588

Owner

stephenmcd commented Mar 5, 2013

Nice! I added a link to this thread from the README so people can check it out if they're interested:

ccb9588

@robert-zaremba

This comment has been minimized.

Show comment
Hide comment
@robert-zaremba

robert-zaremba Mar 16, 2013

@alexvizor: Interesting! Did you found why redis outperforms zmq on single node? In both cases broker uses only one thread (redis and go zmq).
Can I also understand, that this result shows (on single node) that redis 2.6 brings huge speed up in contrast to redis 2.4?

@alexvizor: Interesting! Did you found why redis outperforms zmq on single node? In both cases broker uses only one thread (redis and go zmq).
Can I also understand, that this result shows (on single node) that redis 2.6 brings huge speed up in contrast to redis 2.4?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment