Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stop using multiprocessing.dummy.Pool #126

Merged
merged 3 commits into from Oct 18, 2014
Merged

Stop using multiprocessing.dummy.Pool #126

merged 3 commits into from Oct 18, 2014

Conversation

@untitaker
Copy link
Member

@untitaker untitaker commented Oct 15, 2014

This implements a custom job queue with workers based on threads.
Collection discovery is now done in a separate thread.

See #124

  • Fix tests
  • Document max_workers
  • Deprecate processes
  • Don't sync collections more than once
    • Add tests
@untitaker
Copy link
Member Author

@untitaker untitaker commented Oct 15, 2014

@mistotebe I think this would fix your issue. It reuses the storage classes from the discovery stage for actual syncing.

@untitaker untitaker force-pushed the issue124 branch 6 times, most recently from 7ce54d2 to 0c2bd13 Oct 16, 2014
untitaker added 3 commits Oct 15, 2014
- Custom job queue with workers based on threads.
- Collection discovery is now done in a separate thread. Due to the
  gained flexibility, we could do the sync actions in separate threads
  too?
- The processes parameter has been removed, the related new option is
  only available on the CLI.
@untitaker untitaker force-pushed the issue124 branch from b08ffc0 to aba0a40 Oct 16, 2014
untitaker added a commit that referenced this pull request Oct 18, 2014
Stop using multiprocessing.dummy.Pool
@untitaker untitaker merged commit 6ac71e0 into master Oct 18, 2014
1 check passed
1 check passed
continuous-integration/travis-ci The Travis CI build passed
Details
@untitaker untitaker deleted the issue124 branch Nov 2, 2014
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

1 participant