Join GitHub today
Message queue keeps increasing in ZMQ_CONFLATE sub socket #3171
When setting a SUB socket to use ZMQ_CONFLATE, I expected the message queue to hold only one newest message, and free the old one. But in a test case where a PUB program is sending messages in a loop, and a SUB program sits there collecting these messages, the SUB program collects memory. Some discussion on the mailing list here: https://lists.zeromq.org/pipermail/zeromq-dev/2018-June/032561.html
Ubuntu 16.04, ZMQ 4.2.4
Minimal test code / Steps to reproduce the issue
What's the expected result?
I would have expected that when ZMQ_CONFLATE is set, that the queue doesn't collect messages, and does what it says: only collect the most recent message and free/replace old with the new message.
James Harvey, on mailing list thread, mentioned that in config.hpp, I can compile libzmq with
I am unsure if this is a bug, since I expected ZMQ_CONFLATE to not build up memory like a normal SUB socket would. Either this is something that can be fixed for ZMQ_CONFLATE sockets, or there is some setting/configuration/behavior that I should be using that I don't know about
Adding a little more info from my investigation.
The memory rises until
Its an odd use case but maybe there should be a timer that can trigger the
Thanks @laplaceyang, that makes sense and it allows the caller to control how often they want the queue to be conflated.
I tried it out and it works like a charm, here is the test code which forces the memory prune to make the results clear. Even with heavy publisher the memory never climbs above the baseline.