AssertionError if i try to create new pool after old_pool.close()

[ivanmarchenko83]

this code produces AssertionError:

p = Pool(5)
p.map(local_func, params_chunk)
p.close()
p.join()

....

p = Pool(5)
p.map(local_func, params_chunk)
p.close()
p.join()

so basically if i do close and join i can't use Pool() anymore, even if that is in the different object - pretty strange, probably some static variables issue.

[mmckerns]

I don't see an AssertionError from similar test code. Can you please provide some self-contained small example that produces the behavior you are seeing? Also provide your version of python and of pathos.

[mmckerns]

Note that a Pool in pathos is a singleton, so if you close a Pool and try to use it again it is designed to fail unless you also clear if from memory. Each unique pool has an ID that is registered in a global hashmap.

Here's the expected behavior:

Python 3.8.18 (default, Aug 25 2023, 04:23:37) 
[Clang 13.1.6 (clang-1316.0.21.2.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import pathos.pools as pp
>>> p = pp.ProcessPool(4) # create a pool
>>> p.map(lambda x:x*x, range(4))
[0, 1, 4, 9]
>>> p.close()
>>> p.join()
>>> 
>>> p = pp.ProcessPool(4) # grabs the existing closed pool
>>> p.map(lambda x:x*x, range(4))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/mmckerns/lib/python3.8/site-packages/pathos/multiprocessing.py", line 154, in map
    return _pool.map(star(f), zip(*args), **kwds)
  File "/Users/mmckerns/lib/python3.8/site-packages/multiprocess/pool.py", line 364, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/Users/mmckerns/lib/python3.8/site-packages/multiprocess/pool.py", line 473, in _map_async
    self._check_running()
  File "/Users/mmckerns/lib/python3.8/site-packages/multiprocess/pool.py", line 350, in _check_running
    raise ValueError("Pool not running")
ValueError: Pool not running
>>> p.clear() # clear the pool from memory
>>> p.map(lambda x:x*x, range(4))
[0, 1, 4, 9]
>>> p.close()
>>> p.join()
>>> p.clear() # clear the pool from memory
>>> 
>>> p = pp.ProcessPool(4)
>>> p.map(lambda x:x*x, range(4))
[0, 1, 4, 9]
>>> p.close()
>>> p.join() # not cleared from memory
>>> 
>>> p = pp.ProcessPool(2) # create a different new pool
>>> p.map(lambda x:x*x, range(4))
[0, 1, 4, 9]
>>> p.close()
>>> p.join()
>>> p.clear()
>>> 

[ivanmarchenko83]

Thanks for you help, the problem was that I was not doin p.clear() - for some reason did not see it anywhere in examples
I am using it in python 2.7 and instead of ValueError: Pool not running in your example was getting AssertionError