Exception in error handler leads to FFI garbage collection fatal Python error

[HelloZeroNet]

• gevent version: 1.4.0
• Python version: cPython 3.7.3 downloaded from python.org
• Operating System: Win10

Description:

I want to suppress handled errors happened in threads.

Hub error: FileNotFoundError
Greenlet error: [Errno 2] No such file or directory: 'nonexistent.txt'
Fatal Python error: ffi.from_handle() detected that the address passed points to garbage. If it is really the result of ffi.new_handle(), then the Python object has already been garbage collected

Thread 0x0000939c (most recent call first):
  File "C:\Python3\lib\site-packages\gevent\_threading.py", line 84 in wait
  File "C:\Python3\lib\site-packages\gevent\_threading.py", line 166 in get
  File "C:\Python3\lib\site-packages\gevent\threadpool.py", line 270 in _worker

Current thread 0x00001e7c (most recent call first):
  File "C:\Python3\lib\site-packages\gevent\_ffi\loop.py", line 165 in python_handle_error
  File "C:\Python3\lib\site-packages\gevent\libuv\loop.py", line 445 in __run_queued_callbacks
  File "C:\Python3\lib\site-packages\gevent\libuv\loop.py", line 191 in _run_callbacks
  File "C:\Python3\lib\site-packages\gevent\_ffi\loop.py", line 266 in python_prepare_callback
  File "C:\Python3\lib\site-packages\gevent\libuv\loop.py", line 473 in run
  File "C:\Python3\lib\site-packages\gevent\hub.py", line 582 in run
[Finished in 0.5s with exit code 3221226505]

What I've run:

import gevent.monkey
import gevent.threadpool
import sys

gevent.monkey.patch_all()

pool = gevent.threadpool.ThreadPool(1)

def handleError(self, context, type, value, tb):
    print("Hub error:", sys.exc_info()[0].__name__)

gevent.hub.Hub.handle_error = handleError

def openFile():
    open("nonexistent.txt")

try:
    gevent.spawn(openFile).get()
except Exception as err:
    print("Greenlet error:", err)

try:
    pool.apply(openFile)
except Exception as err:
    print("Pool err", err)

I can get rid of the fatal error by avoid using sys.exc_info() (It does not work if error happens in other thread), but it took me a while realizing why the crash is happening.

Also looking for better way to suppress caught errors printing to output other than overriding gevent.hub.Hub.handle_error

[jamadden]

Also looking for better way to suppress caught errors printing to output other than overriding gevent.hub.Hub.handle_error

Please don't assign to the class. It is documented that you can assign to the instance, e.g., gevent.get_hub().handle_error = handleError.

def handleError(self, context, type, value, tb):
    print("Hub error:", sys.exc_info()[0].__name__)

Please do use the provided type, value, and traceback arguments. They're there for a reason.

Making those minor fixes to the program solves the problem:

import gevent.monkey
import gevent.threadpool

gevent.monkey.patch_all()

pool = gevent.threadpool.ThreadPool(1)

def handleError(context, t, v, tb):
    print("Hub error:", t, v, tb)

gevent.get_hub().handle_error = handleError

def openFile():
    open("nonexistent.txt")

try:
    gevent.spawn(openFile).get()
except Exception as err:
    print("Greenlet error:", err)

try:
    pool.apply(openFile)
except Exception as err:
    print("Pool err", err)

$ GEVENT_LOOP=libuv python /tmp/foo.py
Hub error: <class 'FileNotFoundError'> [Errno 2] No such file or directory: 'nonexistent.txt' <traceback object at 0x1088caeb0>
Greenlet error: [Errno 2] No such file or directory: 'nonexistent.txt'
Hub error: <class 'FileNotFoundError'> [Errno 2] No such file or directory: 'nonexistent.txt' <traceback object at 0x107b14370>
Pool err [Errno 2] No such file or directory: 'nonexistent.txt'

Note that there is the tuple NOT_ERROR that you can assign to as well. It's commonly set globally (i.e., on the class) for particular applications, but you can also set it on an instance. If you always want to ignore some exceptions, that's the best way to do it.

import gevent.monkey
import gevent.threadpool

gevent.monkey.patch_all()

pool = gevent.threadpool.ThreadPool(1)

gevent.get_hub().NOT_ERROR += (FileNotFoundError,)

def openFile():
    open("nonexistent.txt")

try:
    gevent.spawn(openFile).get()
except Exception as err:
    print("Greenlet error:", err)

try:
    pool.apply(openFile)
except Exception as err:
    print("Pool err", err)

$ GEVENT_LOOP=libuv python /tmp/foo.py
Greenlet error: [Errno 2] No such file or directory: 'nonexistent.txt'
Pool err [Errno 2] No such file or directory: 'nonexistent.txt'

[HelloZeroNet]

Thanks for the suggestions, but it does not solves the topic starter issue: if there is an error in the handleError function, then it will lead to Fatal Python error instead of "During handling of the above exception, another exception occurred:"

[jamadden]

That's correct. Don't raise exceptions from there. That's low-level functionality called when the system is in an uncertain state.