Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

save_state not working with multicore #155

Open
MihaiBabiac opened this issue May 4, 2022 · 0 comments
Open

save_state not working with multicore #155

MihaiBabiac opened this issue May 4, 2022 · 0 comments

Comments

@MihaiBabiac
Copy link

It's impossible to run the code with both --multicore and --save_state at the same time. The reason is that the code tries to pickle the process pool, which is not possible, leading to a crash.

See below my fix for this issue.

--- a/PonyGE2/src/utilities/algorithm/state.py
+++ b/PonyGE2/src/utilities/algorithm/state.py
@@ -1,6 +1,9 @@
 import pickle
 import random
 from os import path
+from multiprocessing import Pool
+
+from utilities.algorithm.initialise_run import pool_init
 
 
 def create_state(individuals):
@@ -32,6 +35,10 @@ def create_state(individuals):
     pickle_params = {param: (check_name(params[param]) if callable(
         params[param]) else params[param]) for param in params}
 
+    # the process pool is not picklable, so we don't save it
+    if "POOL" in pickle_params:
+        del pickle_params["POOL"]
+
     # Create a picklable version of the trackers module.
     pickle_trackers = {i: getattr(trackers, i) for i in dir(trackers)
                        if not i.startswith("__")}
@@ -123,6 +130,11 @@ def set_state(state):
     for param in state['params']:
         params[param] = state['params'][param]
 
+    if params['MULTICORE']:
+        # initialize pool if multi-core is enabled
+        params['POOL'] = Pool(processes=params['CORES'], initializer=pool_init,
+                              initargs=(params,))
+
     # Set correct param imports for specified function options, including
     # error metrics and fitness functions.
     set_param_imports()
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant