Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stop using multiprocessing.dummy.Pool #126

Merged
merged 3 commits into from Oct 18, 2014
Merged

Stop using multiprocessing.dummy.Pool #126

merged 3 commits into from Oct 18, 2014

Conversation

untitaker
Copy link
Member

This implements a custom job queue with workers based on threads.
Collection discovery is now done in a separate thread.

See #124

  • Fix tests
  • Document max_workers
  • Deprecate processes
  • Don't sync collections more than once
    • Add tests

@untitaker
Copy link
Member Author

@mistotebe I think this would fix your issue. It reuses the storage classes from the discovery stage for actual syncing.

@untitaker untitaker force-pushed the issue124 branch 6 times, most recently from 7ce54d2 to 0c2bd13 Compare October 16, 2014 19:04
- Custom job queue with workers based on threads.
- Collection discovery is now done in a separate thread. Due to the
  gained flexibility, we could do the sync actions in separate threads
  too?
- The processes parameter has been removed, the related new option is
  only available on the CLI.
untitaker added a commit that referenced this pull request Oct 18, 2014
Stop using multiprocessing.dummy.Pool
@untitaker untitaker merged commit 6ac71e0 into master Oct 18, 2014
@untitaker untitaker deleted the issue124 branch November 2, 2014 21:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant