First [uber hacky] attempt to address tpot issue #95: parallelizing t… #100
Conversation
…allelizing the pipeline evaluations.
|
Cool beans! That's all it took to parallelize DEAP? Very curious to see how effectively it shares memory. I'm rushing against paper deadlines again, but I'm very eager to tinker with this PR to see if we can merge it into TPOT-master. My main concerns are:
|
|
Going to close this PR since we won't be able to merge it into the existing dev branch, but thank you for putting the demo together. We will refer to this PR when considering parallelization options for TPOT. |
|
Sorry for dropping off; job took over and hasn't let up. However I am working on designing additional experiments for a paper that will involve testing multiple ML pipelines, so if teaching this semester can become somewhat self-sustaining I may be able revisit this issue sooner rather than later.
|
|
Totally understand! If you end up putting together another PR, please PR it to the |
|
Will do, thanks!
|
|
Hi guys, is there any progress on this PR? I'm playing now with tpot and paralellizing it on DEAP level would be great. May I ask why what was the reason that prevented you from merging magsol's PR into the dev branch? If I pick this up, some background knowledge could help a lot. Thanks! |
|
@mficek The reason was pretty straightforward: I ran out of free time to run all the experiments that would empirically show one way or another whether this particular strategy of parallelization yielded any benefit :) With spring break coming up it's possible I may be able to come back to this, especially if I know someone is actively interested. |
|
Check the development branch of TPOT. @weixuanfu2016 implemented multiprocessing for TPOT there, although I'll note that it's still not fully validated. |
|
Yes, I tried @weixuanfu2016's version yesterday, it looks great! And now I see it's even in the development branch, sweet. |
This is a first attempt to parallelize the GA pipeline in tpot. Since tpot uses DEAP under the hood for the parallel algorithm, the parallelization strategy involves overriding its
mapimplementation.The DEAP documentation specifies two methods for parallelization: SCOOP and Python's built-in
multiprocessinglibrary. The former is not feasible, given the entire package has to be run through it. The latter is possible, but there is a serialization error by which DEAP attempts to pickle theToolboxobject and, therefore, the encapsulatedPoolobject (this error).The workaround implemented in this PR is to extend the DEAP ToolBox to include required
__getstate__and__setstate__methods that eliminate thePoolobject just prior to serialization.Where should the reviewer start?
The parallelization appears to work (see screenshot), but it's unclear if a whole separate subclass is required.
joblibwould be preferable here, but there doesn't seem to be a way to decouple the partial it generates from its dispatcher and hand that over to DEAP.How should this PR be tested?
Performance testing, primarily. From the more knowledgeable committers, a deeper dive into whether or not subclassing is really the answer here.
Any background context you want to provide?
Mostly provided above. This is not ready for inclusion; several other important points remain, mainly a command-line parameter to control for parallelization. I just wanted to get the review process rolling.
What are the relevant issues?
#95
Screenshots (if appropriate)
## Questions: - Do the docs need to be updated? Yes, we'll need to add a command-line parameter to control parallelization. - Does this PR add new (Python) dependencies? No, just a new module.