Question on RuntimeError

[gdebure]

Hi there,

I've been using pebble for a while, mainly for its timeout capabilities compared to plain concurrent.futures.

Lately I've run into situations where the following runtime error is raised:

File "/infinite/venus/src/task_scheduler/process_scheduler.py", line 231, in _schedule
    return self._executor.schedule(start_job, kwargs={"job": job}, timeout=self._timeout)
  
File "/usr/local/lib/python3.7/site-packages/pebble/pool/process.py", line 85, in schedule
    self._check_pool_state()

File "/usr/local/lib/python3.7/site-packages/pebble/pool/base_pool.py", line 94, in _check_pool_state
    raise RuntimeError('Unexpected error within the Pool')

RuntimeError: Unexpected error within the Pool

I could not find a consistent scenario to trigger it. Sometimes it happens after 1700 jobs have already been scheduled (all of the same type), sometimes everything goes fine even with more than 32000 jobs scheduled.

I'm a bit blind on what could go wrong, I'd appreciate some pointers as to where I should look for... What are the situations that are likely to trigger this ?

Thanks :)

[noxdafox]

Hi gdebure,

Pebble's pool relies on 3 threads to maintain its internal logic. Every time you interact with the Pool APIs the health of the 3 threads is checked and if one or more are not running, you get the RuntimeError you see.

If one of the internal threads crashes, it will print out the stack trace with the actual reason for the crash. I'd suggest you to search within your app logs, you should be able to see the actual reason for the crash.

My blind guesses are two: either you run low on memory and one of the threads could not allocate any further memory, or the Pipe used to communicate with the worker processes was closed for some reason.

Once you find the traceback of the actual crash, please paste it here so I can provide further hints.

[noxdafox]

Closing this ticket due to inactivity, please re-open if you still have issues.

[NikitaKononov]

Closing this ticket due to inactivity, please re-open if you still have issues.

Hello, I've just faced same issue
RuntimeError: Unexpected error within the Pool
With any number of threads between 1 and 24

I don't have any other traceback. Does pebble surpress errors from executing code?
P.S. Not running on low memory