Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove queue prefetching from component service queues #749

Closed
jrobble opened this issue Nov 30, 2018 · 0 comments
Closed

Remove queue prefetching from component service queues #749

jrobble opened this issue Nov 30, 2018 · 0 comments

Comments

@jrobble
Copy link
Member

jrobble commented Nov 30, 2018

Prefetching behavior can result in starving services in certain situations, resulting in inefficiencies.

For example, If two services are available to process two sub-jobs (one message in the queue for each), the first service may start consuming one sub-job and ActiveMQ may dispatch the next sub-job to the same service as part of the pre-fetching behavior. The second service is then starved. Optimally, the second service should be processing a sub-job at the same time as the first service.

A similar situation happens when only one service is running and a second service becomes available after ActiveMQ dispatches the second sub-job to the first service as part of pre-fetching. Users will often bring up new service instances to distribute the workload. The workload will not be distributed in this scenario. Once this issue is resolved it should.

Note that in the past setting pre-fetch to 0 caused ActiveMQ/messing errors/issues. Be on the lookout for those.

This issue is related to #647.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Closed
Development

No branches or pull requests

3 participants