fails to bubble up maximum recursion depth errors raised from greenlets

[danmilon]

• gevent version: 20.9.1
• Python version: cPython 2.7.18, but it's present in 3.8.5 too
• Operating System: Fedora 32 Linux 5.8.10-200.fc32.x86_64

Description:

When gevent tries to bubble up an exception raised in a greenlet (Greenlet.__report_error()), if when trying to get a dump of the traceback object another exception is raised, the greenlet goes into a broken state.

For example, in the following code:

import gevent

def _test():
    while True:
        gevent.sleep(1)
        print("I'm alive.")

# spawn something to prevent gevent from complaining about blocking forever
# with the recursive greenlet.
test_g = gevent.spawn(_test)

def recursive():
    recursive()

r1 = gevent.spawn(recursive)

print("Before get.")
r1.get() # this never returns
print("After get.")

It spawns a greenlet that goes into an infinite recursion and eventually raises a RuntimeError.

r1.get() would be expected to re-raise that, but instead it blocks forever, because when trying to dump_traceback() inside Greenlet.__repor_error() it gets another exception.

Traceback (most recent call last):
  File "src/gevent/greenlet.py", line 856, in gevent._gevent_cgreenlet.Greenlet.run
  File "src/gevent/greenlet.py", line 837, in gevent._gevent_cgreenlet.Greenlet._Greenlet__report_error
  File "/home/danmilon/tmp/gevent/src/gevent/_tblib.py", line 415, in g
    return f(a)
  File "/home/danmilon/tmp/gevent/src/gevent/_tblib.py", line 471, in dump_traceback
    return dumps(tb)
  File "/home/danmilon/.pyenv/versions/2.7.18/lib/python2.7/copy_reg.py", line 74, in _reduce_ex
    getstate = self.__getstate__
RuntimeError: maximum recursion depth exceeded while calling a Python object
2020-11-23T13:06:10Z <callback at 0x7fe4304846e0 stopped> failed with RuntimeError

And because __report_error() dies midway, no exception is ever raised to r1.get().

Maybe it's not possible to pickle the maximum recursion traceback?
Or the dump_traceback() call in __report_error() should be wrapped in a try/except to prevent unexpected errors from breaking gevent.

BTW the same happens in cPython 3.8.5 too:

$ python ~/tmp/gevent-bug.py 
Before get.
Traceback (most recent call last):
  File "src/gevent/greenlet.py", line 854, in gevent._gevent_cgreenlet.Greenlet.run
  File "/home/danmilon/tmp/gevent-bug.py", line 13, in recursive
    recursive()
  File "/home/danmilon/tmp/gevent-bug.py", line 13, in recursive
    recursive()
  File "/home/danmilon/tmp/gevent-bug.py", line 13, in recursive
    recursive()
  [Previous line repeated 993 more times]
RecursionError: maximum recursion depth exceeded

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "src/gevent/greenlet.py", line 856, in gevent._gevent_cgreenlet.Greenlet.run
  File "src/gevent/greenlet.py", line 837, in gevent._gevent_cgreenlet.Greenlet._Greenlet__report_error
  File "/tmp/tmp.CObOiHqW0D/venv/lib/python3.8/site-packages/gevent/_tblib.py", line 415, in g
    return f(a)
  File "/tmp/tmp.CObOiHqW0D/venv/lib/python3.8/site-packages/gevent/_tblib.py", line 470, in dump_traceback
    return dumps(tb)
  File "/tmp/tmp.CObOiHqW0D/venv/lib/python3.8/site-packages/gevent/_tblib.py", line 133, in __getattr__
    return self[name]
RecursionError: maximum recursion depth exceeded while calling a Python object
2020-11-23T13:12:27Z <callback at 0x7ff1d591be00 stopped> failed with RecursionError

I'm alive.

[danmilon]

Oh great, thanks :)

[damz]

@jamadden Thanks for the quick fix, very appreciated.