-
Notifications
You must be signed in to change notification settings - Fork 449
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Expose size of the push manager's queue #20
Comments
I totally agree with this one 👍 I have a SQS queue already in-front of the apns push sending artifacts, and the current unbounded in-memory queue would mean draining it and having it all in memory. |
You could use a Guava RateLimiter[1] to limit the rate of adding messages https://gist.github.com/marcomorain/7470870#file-ratelimitedpushmanager-java On Thu, Nov 14, 2013 at 4:12 PM, Francisco A. Lozano <
|
@danmassie Thanks kindly for both the suggestion and the pull request! I'm hesitant on this one because the queue size can fluctuate quite a bit during normal operation. Here's a situation I worry about:
Generally, I'm worried that the number of notifications in the queue isn't super meaningful until we've shut down (see #4), and just exposing the queue's size could lead to a false sense of certainty about the state of things. If the goal is to prevent memory usage from getting out of control, though, I wonder if there's something else we can try. To be clear, though, I would do this in a heartbeat (and would have already done it) if we could be more certain about the fate of non-rejected notifications. |
I think that exposing the size of the queue as an indicator of the size is better than nothing at all. I appreciate that it can change, but that is the nature of the process. |
So I've been thinking about this and have a plan that I like a bit better than exposing the internal queue size. Right now, we have one queue that does everything. New messages from the outside go into that queue, and so do messages that need to be resent. What I'd like to do is break that into two queues: an "interacts with the outside world" queue and a retry queue. The former -- the queue that accepts messages from the outside -- could optionally be provided by callers at construction time. If you wanted to, you could provide a bounded queue implementation. The plan would be to make sure the retry queue is empty before taking new messages from the public-facing queue. That would mean you could be reasonably confident that things were moving forward if the public-facing queue started to drain. You could monitor the public queue and even alter its contents without fear of bad things happening to Pushy's internals. I think that would also help organize things in such a way that we could consider some robust solutions to #24, too. Sound like that would solve your problem? |
When adding messages asynchronously, the entire list of messages (~200k) is added to the queue almost immediately. Whilst this is ok just now, it would be nice to be able to choose to wait to enqueue more messages once the size of the queue had reduced to an acceptable size. Exposing the size of the queue would allow this to happen and would help prevent it growing too large, which could result in out of memory errors.
The text was updated successfully, but these errors were encountered: