Lower Level Poller Pattern Question

[peske]

In NetMQ poller is obviously implemented in NetMQPoller class, as "event-driven reactor". It's definitely easier pattern than original ZeroMQ implementation, but I've ran into a situation where I need lower level pattern (like mspoller from ZeroMQ guide). Basically I need to get a message (single message) from any of two receiver sockets (or to wait up to timeout for one to arrive). After that the thread should do some other work, and repeat cycle again (getting message -> other work), in an infinite loop. I hope that you understand what I need - it's actually the same exact thing as the linked example from above.

Does NetMQ offer some lower-level poller API that would allow this? Any alternative approach?

Thanks!

[somdoron]

We used to have something like that in the old poller, anyway I suggest two solutions:

1. Make a pull request to netmq and add a RunOnce method which will receive a timeout and will run until timeout or first receive ready.
2. Wrap netmqpoller, should be pretty simple, each receive ready should immediately cancel the poller, you can also have netmqtimeout, which also cancel the poller, use the sync run version.

[peske]

@somdoron Thanks for replying so fast!

Regarding the first suggestion: I'm still beginner with NetMQ, and I'm not feeling comfortable enough to PR, but if I come up with something good - I'll share. There's another thing that worries me about this approach: implementing lower-level API (zmq_poll wrapper) by further building upon the existing higher level API (NetMQPoller) does not sound as the right thing to do. Things are usually done in the opposite direction: create performant lower level API, and then build the higher level upon it. It is especially true if we are ware of the fact that NetMQPoller can be performance expensive, and especially in projects like NetMQ where performances are crucial.

About the second approach I'll have to ask you for advice: We are talking about the system that should be able to handle 20,000+ messages per second. It means that I should start / stop poller 20,000+ times per second. Do you still think I can go this way?

Thank you very much for trying to help!

[peske]

I've tried, but I'm far from understanding NetMQPoller.cs, so I'll give up on creating pr with RunOnce. I'll have to try with the second suggested approach.

In any case I suggest you to consider adding some light-weight version of zmq_poll. NetMQPoller is obviously too massive (and too slow), and at the moment it is the only way to access some kind of polling.

[drewnoakes]

Here's the old Poller class at the time NetMQPoller was introduced, including PollOnce:

https://github.com/zeromq/netmq/blob/932bf9d3e31090aa33ab495050c7b8f5a48a7ddb/src/NetMQ/Poller.cs

NetMQPoller has a lot in common with the Poller from that time, and was merged with (the now removed) NetMQScheduler, giving the class you've been looking at today.

Would that implementation be light-weight enough for your requirement? It still uses much of the same machinery underneath.

[peske]

Thanks for trying to help!

As far as I can see in the code, there would be no difference in performances if I implement RunOnce comparing to using the existing implementation and calling Stop as soon as the first event happen. So I'll do the later.

I'm still afraid a bit that NetMQPoller class will be a bottleneck, and I'll compare performances with clrzmq implementation of zmq_poll when I have bit more time, and let you know about the results.

Thanks again!

[peske]

I've ran into problems with NetMQPoller and opened another issue: #598

[stale]

This issue has been automatically marked as stale because it has not had activity for 365 days. It will be closed if no further activity occurs within 56 days. Thank you for your contributions.

[poroburu]

Signalling my interest in a NetMQTimeout.

Working through the zguide in F# using FsNetMQ. fszmq is now deprecated.
Managed to work around C style ZMQ_POLL using Poller. Almost reaching F# example parity at the end of chapter 3 until peering2, then the polling logic got a bit complicated.