Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added getRetryQueueSize #114

Closed
wants to merge 1 commit into from
Closed

Added getRetryQueueSize #114

wants to merge 1 commit into from

Conversation

fmmr
Copy link

@fmmr fmmr commented Aug 30, 2014

to be able to monitor the retryqueue from outside this class. Also added getQueueSize for symmetry.

…side this class. Also added getQueueSize for symmetry.
@jchambers
Copy link
Owner

Why is it important to know the size of the retry queue?

@jchambers
Copy link
Owner

…to be clear, I'm not asserting that it isn't important. I just don't understand the use case here.

@fmmr
Copy link
Author

fmmr commented Sep 1, 2014

I think it's worth-while to have a feeling of what's going on. I monitor our application every 30 seconds with lots of queues.
We've had trouble with out-of-memory errors due to unbounded queues. As the retry-queue is such a queue, I think it's worth while being able to monitor it.

@jchambers
Copy link
Owner

We've been talking about this a bit; we think there is a real issue to be addressed here, since there's no way of knowing what happened to a push notification after it was taken from the public queue. That said, a notification can be in a number of different places after it leaves the public queue and still be "in play." They are:

  1. The retry queue, as you've pointed out here.
  2. The Netty machinery (in a Callable or Runnable or Future) that actually attempts to write notifications.
  3. The sent notification buffer; notifications in the sent notification buffer might get moved to the retry queue if a prior notification is rejected by the APNs gateway.

Just exposing the retry queue size, then, only gives part of the picture. Our leaning right now is to add a method roughly along the lines of getEstimatedNumberOfUnconfirmedNotifications that returns the sum of the notifications in all of those places. It's far from perfect, but I think it's the best we can do with the tools at hand.

So, in short, I think you're barking up the right tree, but this isn't quite how we'd like to solve the problem. Let us think on it a little more. We'll take responsibility for implementing whatever we settle on.

@jchambers jchambers added this to the v0.5 milestone Oct 1, 2014
@jchambers
Copy link
Owner

For posterity, this came up in the context of a mailing list discussion about an out-of-memory error.

@jchambers
Copy link
Owner

#139 also addresses a similar issue.

Carrying over the discussion from #139, I'd argue that while the retry queue is technically unbounded, it is practically bounded because each connection's outbound buffer has a high-water mark of 64kB. In theory, if every connection dumped all of its unacknowledged notifications into the retry queue at the same time, you'd still be bounded by (very roughly) 64kB * nConnections. (EDIT: sorry—that's not quite right. It'd be (64kB + sizeOfSentNotificationBuffer) * nConnections).

We addressed an out-of-memory issue in #138; is it possible that was the problem you were seeing? Also, what action would you take based on the size of the retry queue? Would it be significantly different than using a bounded queue implementation for the public queue?

Thanks for your patience!

@jchambers jchambers removed this from the v0.5 milestone May 24, 2015
@jchambers
Copy link
Owner

@fmmr I think this is a great suggestion, and we really appreciate the pull request. I think we're going to pass on this specific implementation for now in favor of doing something more robust (JMX, maybe?) as described in #175.

Thanks kindly for the contribution!

@jchambers jchambers closed this May 24, 2015
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants