Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement Basic Parallel Capabilities #171

Closed
mandli opened this issue Dec 23, 2015 · 1 comment
Closed

Implement Basic Parallel Capabilities #171

mandli opened this issue Dec 23, 2015 · 1 comment

Comments

@mandli
Copy link
Member

mandli commented Dec 23, 2015

Propose to add a simple process pool so that multiple workers can take frames until plotting is done rather than use a single process.

Add code into the plotdata object to enable parallel running and parallelize the
png file creation by launching multiple processes that will take jobs as they
finish with each frame.

  • Not sure if this will really help as plotting may be a memory problem rather than a compute problem
  • This is probably not as efficient as it could be but something better would probably require a lot more refactoring
  • Future design should encapsulate better a single frame (or figure's) plotting process to break this down into even more work units
  • Multiprocessing has issues with pickle due to simple lambda implementations
  • May be able to provide initialization but this would require a refactor of plotpages.plotclaw_driver
@mandli
Copy link
Member Author

mandli commented Oct 1, 2016

Basic functionality added in #174.

@mandli mandli closed this as completed Oct 1, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant