BlockingIOError: [Errno 11] Resource temporarily unavailable

[francoislaurent]

If you hit the following error message:

  File "/usr/local/lib/python3.6/dist-packages/tramway/inference/optimization.py", line 1153, in minimize_sparse_bfgs1
    sched.run()
  File "/usr/local/lib/python3.6/dist-packages/tramway/core/parallel/__init__.py", line 441, in run
    w.start()
  File "/usr/lib/python3.6/multiprocessing/process.py", line 105, in start
    self._popen = self._Popen(self)
  File "/usr/lib/python3.6/multiprocessing/context.py", line 223, in _Popen
    return _default_context.get_context().Process._Popen(process_obj)
  File "/usr/lib/python3.6/multiprocessing/context.py", line 277, in _Popen
    return Popen(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 19, in __init__
    self._launch(process_obj)
  File "/usr/lib/python3.6/multiprocessing/popen_fork.py", line 66, in _launch
    self.pid = os.fork()
BlockingIOError: [Errno 11] Resource temporarily unavailable

in a Slurm-submitted job, you may have failed to pass the proper sbatch options.

As of version 0.5 , this is known to happen in combination with the 'stochastic.dv' inference module. Please check that you specified the worker_count argument of the mapper attribute AND reflected this value in the sbatch_options argument of the corresponding pipeline stage.

For example:

a = RWAnalyzer()
...
a.mapper.from_plugin('stochastic.dv')
a.mapper.worker_count = 20 # number of processes (minus 1) that will run on a node
...
a.env = environments.SlurmOverSSH # or derived classes
...
def inference_stage(self):
   ...
   self.mapper.infer(...)
...
a.pipeline.append_stage(inference_stage, granularity='roi',
       sbatch_options=dict(c=a.mapper.worker_count+1)) # <- PLEASE NOTE
...
a.run()

As some increased degree of automation is desired, this issue will stay open as long as explicitly specifying sbatch_options is necessary in the above example.