Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use cast instead of call to enqueue #5

Merged
merged 1 commit into from
Jul 30, 2018
Merged

Conversation

rhnonose
Copy link
Contributor

When there's an increased amount of workers enqueing work on the feeder, using a GenServer.call is not realistic, even with a really high timeout, and a response is not strictly required. This PR changes call to cast for stop, resume, pause and enqueue.

@coveralls
Copy link

Coverage Status

Coverage decreased (-0.07%) to 98.148% when pulling 1940c31 on rhnonose:master into b635913 on fredwu:master.

@fredwu
Copy link
Owner

fredwu commented May 27, 2018

Thanks @rhnonose! Is this related to fredwu/crawler#11?

@rhnonose
Copy link
Contributor Author

@fredwu It's unrelated.

@fredwu
Copy link
Owner

fredwu commented Jun 5, 2018

Hmm, even if you set the timeout to infinity? The reason for using call was if an enqueue'd job crashes the queue should know about it. 🤔

@rhnonose
Copy link
Contributor Author

rhnonose commented Jun 5, 2018

I think that's gonna hang all the processes until it exhausts memory, at least the way it's used in the crawler.

Since the crawler AFAIK runs a breadth-first on the links, the processes waiting will increase exponentially (speculating).

But thinking more for opq itself as a stand-alone lib, I'm not sure. call with timeout to infinity will only make it a tighter bottleneck IMO.

@fredwu fredwu merged commit 0aafb01 into fredwu:master Jul 30, 2018
@fredwu
Copy link
Owner

fredwu commented Jul 30, 2018

Merged, thanks @rhnonose!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants