thread error for nested maps

[mmckerns]

on linux cluster with python 2.6.4, MM2_surrogate_diam_batchgrid.py fails with:

Traceback (most recent call last):
  File "MM2_surrogate_diam_batchgrid.py", line 197, in <module>
    diameter = UQ(RVstart,RVend,lower_bounds,upper_bounds)
  File "MM2_surrogate_diam_batchgrid.py", line 126, in UQ
    results = Pool(nnodes).map(optimize, cf,lb,ub,nb)
  File "/home/mmckerns/lib/python2.6/site-packages/pathos-0.2a1.dev-py2.6.egg/pathos/multiprocessing.py", line 108, in map
    return __STATE['pool'].map(star(f), zip(*args)) # chunksize
  File "/home/mmckerns/lib/python2.6/site-packages/processing-0.52_pathos-py2.6-linux-x86_64.egg/processing/pool.py", line 130, in map
    return self.mapAsync(func, iterable, chunksize).get()
  File "/home/mmckerns/lib/python2.6/site-packages/processing-0.52_pathos-py2.6-linux-x86_64.egg/processing/pool.py", line 373, in get
    raise self._value
thread.error: can't start new thread

similarly, MPI2_surrogate_diam_batchgrid.py fails with:

Traceback (most recent call last):
  File "/home/mmckerns/bin/ezpool.py", line 5, in <module>
    pkg_resources.run_script('pyina==0.2a1.dev', 'ezpool.py')
  File "/home/mmckerns/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/pkg_resources.py", line 489, in run_script
  File "/home/mmckerns/lib/python2.6/site-packages/setuptools-0.6c11-py2.6.egg/pkg_resources.py", line 1207, in run_script
  File "/home/mmckerns/lib/python2.6/site-packages/pyina-0.2a1.dev-py2.6.egg/EGG-INFO/scripts/ezpool.py", line 57, in <module>
    res = parallel_map(func, *args, **kwds) #XXX: called on ALL nodes ?
  File "/home/mmckerns/lib/python2.6/site-packages/pyina-0.2a1.dev-py2.6.egg/pyina/mpi_pool.py", line 65, in parallel_map
    pool = MPool(1) #XXX: poor pickling... use iSend/iRecv instead?
  File "/home/mmckerns/lib/python2.6/site-packages/processing-0.52_pathos-py2.6-linux-x86_64.egg/processing/pool.py", line 100, in __init__
    self._task_handler.start()
  File "/usr/local/python-2.6.4/lib/python2.6/threading.py", line 471, in start
    _start_new_thread(self.__bootstrap, ())
thread.error: can't start new thread
solved: [104.90247657509542, 1.0639249013840923e-10, 2.2856829028068693, 60.000000000004114]
solved: [98.740640792599351, 0.68621813955341593, 2.1000000001085253, 2.7999999999909924]

Both work fine on mac OSX with python 2.7.8 and 2.6.9

[mmckerns]

These don't really work fine… they can have all sorts of unexpected behavior, like leaving alive daemon python child processes. Closing, as per #33.