-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
First [uber hacky] attempt to address tpot issue #95: parallelizing t… #100
Conversation
…allelizing the pipeline evaluations.
Cool beans! That's all it took to parallelize DEAP? Very curious to see how effectively it shares memory. I'm rushing against paper deadlines again, but I'm very eager to tinker with this PR to see if we can merge it into TPOT-master. My main concerns are:
|
Going to close this PR since we won't be able to merge it into the existing dev branch, but thank you for putting the demo together. We will refer to this PR when considering parallelization options for TPOT. |
Sorry for dropping off; job took over and hasn't let up. However I am working on designing additional experiments for a paper that will involve testing multiple ML pipelines, so if teaching this semester can become somewhat self-sustaining I may be able revisit this issue sooner rather than later.
|
Totally understand! If you end up putting together another PR, please PR it to the |
Will do, thanks!
|
Hi guys, is there any progress on this PR? I'm playing now with tpot and paralellizing it on DEAP level would be great. May I ask why what was the reason that prevented you from merging magsol's PR into the dev branch? If I pick this up, some background knowledge could help a lot. Thanks! |
@mficek The reason was pretty straightforward: I ran out of free time to run all the experiments that would empirically show one way or another whether this particular strategy of parallelization yielded any benefit :) With spring break coming up it's possible I may be able to come back to this, especially if I know someone is actively interested. |
Check the development branch of TPOT. @weixuanfu2016 implemented multiprocessing for TPOT there, although I'll note that it's still not fully validated. |
Yes, I tried @weixuanfu2016's version yesterday, it looks great! And now I see it's even in the development branch, sweet. |
This is a first attempt to parallelize the GA pipeline in tpot. Since tpot uses DEAP under the hood for the parallel algorithm, the parallelization strategy involves overriding its
map
implementation.The DEAP documentation specifies two methods for parallelization: SCOOP and Python's built-in
multiprocessing
library. The former is not feasible, given the entire package has to be run through it. The latter is possible, but there is a serialization error by which DEAP attempts to pickle theToolbox
object and, therefore, the encapsulatedPool
object (this error).The workaround implemented in this PR is to extend the DEAP ToolBox to include required
__getstate__
and__setstate__
methods that eliminate thePool
object just prior to serialization.Where should the reviewer start?
The parallelization appears to work (see screenshot), but it's unclear if a whole separate subclass is required.
joblib
would be preferable here, but there doesn't seem to be a way to decouple the partial it generates from its dispatcher and hand that over to DEAP.How should this PR be tested?
Performance testing, primarily. From the more knowledgeable committers, a deeper dive into whether or not subclassing is really the answer here.
Any background context you want to provide?
Mostly provided above. This is not ready for inclusion; several other important points remain, mainly a command-line parameter to control for parallelization. I just wanted to get the review process rolling.
What are the relevant issues?
#95
Screenshots (if appropriate)
![screen shot 2016-03-01 at 4 27 39 pm](https://cloud.githubusercontent.com/assets/135417/13442744/b79d54f0-dfcb-11e5-9526-c3e24c62d838.png)
## Questions: - Do the docs need to be updated? Yes, we'll need to add a command-line parameter to control parallelization. - Does this PR add new (Python) dependencies? No, just a new module.