Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How does mapping work of problems to processes work #57

Closed
gloacai opened this issue Apr 18, 2013 · 1 comment
Closed

How does mapping work of problems to processes work #57

gloacai opened this issue Apr 18, 2013 · 1 comment

Comments

@gloacai
Copy link

gloacai commented Apr 18, 2013

Not really a code bug but I'm curious how this gem is slicing up the problem set. Maybe updating the readme with this information will be helpful to others later

I.e. if I say
Parallel.map((1..100), :in_processes => 20) do | item |
...
end

  • Does this create a worker pool of 20 processes that as they finish an iteration they go and grab the next item and run the loops contents for that item?
  • Does it map a range of values to each process (process 1 gets items 1-5, process 2 gets items 6-10, ...)?
  • Some other way

In my code I'm dealing with a shifting window problem where i have to also change the size of the window so I'm dealing with nested loops. The last iteration of my top loop has the least possible window sizes so it runs the fastest but the first iteration has the most possible window sizes so it runs the slowest. If you're using the first way of splitting up the problem set then it works fine for my code but the second way would mean I'm not getting quite as much performance.

There could be other problems though where the second method would be faster because of the lower overhead.

@grosser
Copy link
Owner

grosser commented Apr 18, 2013

From the readme:
Processes/Threads are workers, they grab the next piece of work when they finish.

@grosser grosser closed this as completed Apr 18, 2013
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants