Look into joblib for parallelizing pipeline evaluations

[rhiever]

joblib should, in theory, make it possible to evaluate the entire TPOT GP population in parallel without making copies of the data set. We should look into using joblib to do this and verify whether this is true.

[bartleyn]

Isn't DEAP supposed to provide some support for parallelizing the evaluation? Or does it make copies of the data set?

[rhiever]

Not sure. I never looked into it much, TBH.

[bartleyn]

I dug around and found this. It seems all we SHOULD need to do is replace the toolbox's map function with one that interfaces with some parallel lib. I've run it once with multiprocessing but get the following:

Traceback (most recent call last):
File "test_tpot.py", line 10, in
tpot.fit(X_train, y_train)
File "/Users/Bartley/Documents/personal_dev/tpot/tpot/tpot/tpot.py", line 256, in fit
stats=stats, halloffame=self.hof, verbose=verbose)
File "/Users/Bartley/anaconda/envs/py35/lib/python3.5/site-packages/deap-1.1.0-py3.5-macosx-10.5-x86_64.egg/deap/algorithms.py", line 149, in eaSimple
fitnesses = toolbox.map(toolbox.evaluate, invalid_ind)
File "/Users/Bartley/anaconda/envs/py35/lib/python3.5/multiprocessing/pool.py", line 260, in map
return self._map_async(func, iterable, mapstar, chunksize).get()
File "/Users/Bartley/anaconda/envs/py35/lib/python3.5/multiprocessing/pool.py", line 608, in get
raise self._value
File "/Users/Bartley/anaconda/envs/py35/lib/python3.5/multiprocessing/pool.py", line 385, in _handle_tasks
put(task)
File "/Users/Bartley/anaconda/envs/py35/lib/python3.5/multiprocessing/connection.py", line 206, in send
self._send_bytes(ForkingPickler.dumps(obj))
File "/Users/Bartley/anaconda/envs/py35/lib/python3.5/multiprocessing/reduction.py", line 50, in dumps
cls(buf, protocol).dump(obj)
AttributeError: Can't pickle local object 'TPOT.fit..pareto_eq'

[rhiever]

Try moving the pareto_eq function out of fit() and make it a TPOT class function. Will it work then?

[magsol]

Strange; I'm getting a slightly different error (might be because I'm running tpot.py directly, instead of importing it into an external script).

Traceback (most recent call last):
  File "tpot.py", line 1235, in <module>
    main()
  File "tpot.py", line 1224, in main
    tpot.fit(training_features, training_classes)
  File "tpot.py", line 247, in fit
    stats=stats, halloffame=self.hof, verbose=verbose)
  File "/opt/python/lib/python3.5/site-packages/deap/algorithms.py", line 147, in eaSimple
    fitnesses = toolbox.map(toolbox.evaluate, invalid_ind)
  File "/opt/python/lib/python3.5/multiprocessing/pool.py", line 260, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/opt/python/lib/python3.5/multiprocessing/pool.py", line 608, in get
    raise self._value
  File "/opt/python/lib/python3.5/multiprocessing/pool.py", line 385, in _handle_tasks
    put(task)
  File "/opt/python/lib/python3.5/multiprocessing/connection.py", line 206, in send
    self._send_bytes(ForkingPickler.dumps(obj))
  File "/opt/python/lib/python3.5/multiprocessing/reduction.py", line 50, in dumps
    cls(buf, protocol).dump(obj)
  File "/opt/python/lib/python3.5/multiprocessing/pool.py", line 492, in __reduce__
    'pool objects cannot be passed between processes or pickled'
NotImplementedError: pool objects cannot be passed between processes or pickled

[rhiever]

How are you running it? python -m tpot.tpot out of the main tpot repo directory?

[magsol]

No; changed the one import to from export_utils import * so it wasn't a relative import anymore, and then running it directly as python tpot.py. Should I do it some other way?

[magsol]

@bartleyn how are you structuring your code, i.e. the addition of the Pool.map function to the TPOT toolbox? In my case, it's complaining that the pool object I'm using is attempting to pickle itself.

[bartleyn]

I ended up getting that error as well. I bet you that I have similarly structured code. I've tried a few different iterations on where I instantiate the pool object (inside the class, etc). I've also tried messing around with the TPOT object's dictionary to see if we can ignore pickling the pool object (using the get/set_state functions), but I think I'm barking up the wrong tree, as the TPOT object isn't the object getting pickled.

[rhiever]

@magsol: You won't need to change the export_utils import if you call TPOT as python -m tpot.tpot at the base directory of the repo, e.g.,

cd tpot-master
python -m tpot.tpot dataset.csv -v 2

[magsol]

@rhiever This is just plain bizarre:

$> python -m tpot.tpot dataset.csv -v 2
Version 0.2.7 of tpot is outdated. Version 0.2.8 was released 2 days ago.
Traceback (most recent call last):
  File "/opt/python/lib/python3.5/runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "/opt/python/lib/python3.5/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/tpot/tpot/tpot.py", line 1235, in <module>
    main()
  File "/tpot/tpot/tpot.py", line 1114, in main
    from _version import __version__
ImportError: No module named '_version'

How is it not recognizing the _version module, and yet the version module is spitting out the fact that it's out of date?

[rhiever]

Change the code in tpot.py back to saying

from ._version import __version__

[magsol]

That's what it currently is; if I remove the prepended . I get a different error:

/opt/python/bin/python: Error while finding spec for 'tpot.tpot' (<class 'ImportError'>: No module named '_version')

EDIT Nevermind; stupid mistake on my part. Ignore this post.

[rhiever]

Do you have the latest source? Looks like you have TPOT 0.2.7.

[magsol]

Yep; upgraded. Am now getting the same sequence of errors as @bartleyn (with pareto_eq inside of fit and as a class function).

[rhiever]

phew Progress! Likely need to move pareto_eq outside of the fit() function, then see what else @bartleyn did after that.

[magsol]

First-pass PR incoming. Warning: it's nasty.

[rhiever]

Can't believe how easy that looks when it's all said and done, @magsol! Thank you for figuring that out. I added some comments on the PR directly.

[arita37]

Think there Ipyparalle may solve the issue (add the possibility to work on remote server):
cf here:
https://goo.gl/kgyJMX