Maximum recursion depth exceeded in getattr(). #103272

felixxm · 2023-04-05T09:36:57Z

Bug report

We're hitting RecursionError: maximum recursion depth exceeded in Django test suite with Python 3.12.0a7 when accessing an attribute with a custom __getattr__() method:

ERROR: test_access_warning (deprecation.test_storages.DefaultStorageDeprecationTests.test_access_warning)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/django/tests/deprecation/test_storages.py", line 128, in test_access_warning
    settings.DEFAULT_FILE_STORAGE
  File "/django/django/conf/__init__.py", line 83, in __getattr__
    if (_wrapped := self._wrapped) is empty:
                    ^^^^^^^^^^^^^
  File "/django/django/conf/__init__.py", line 83, in __getattr__
    if (_wrapped := self._wrapped) is empty:
                    ^^^^^^^^^^^^^
  File "/django/django/conf/__init__.py", line 83, in __getattr__
    if (_wrapped := self._wrapped) is empty:
                    ^^^^^^^^^^^^^
  [Previous line repeated 790 more times]
RecursionError: maximum recursion depth exceeded

See affected test and LazySettings.__getattr__().

Bisected to the aa0a73d.

I'd try to prepare a small regression test.

Your environment

CPython versions tested on: Python 3.12.0a7
Operating system and architecture: x86_64 GNU/Linux

Linked PRs

The text was updated successfully, but these errors were encountered:

AlexWaygood · 2023-04-05T16:08:47Z

Bisected to the aa0a73d.

Cc. @wangxiang-hz and @Fidget-Spinner

Fidget-Spinner · 2023-04-05T16:33:34Z

Thanks for finding this. I'll try to debug once there's a minimal reproducer. Sorry I'm a little swamped for the next 2 weeks.

sunmy2019 · 2023-04-06T16:31:01Z

NB: when running this test with debug build, it crashes.

Testing against Django installed in '/home/ubuntu/django-main/django' with up to 8 processes
Found 6 test(s).
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
.python: Python/ceval.c:821: _PyEval_EvalFrameDefault: Assertion `!_PyErr_Occurred(tstate)' failed.
Fatal Python error: Aborted

Current thread 0x00007f2f2750e740 (most recent call first):
  File "/home/ubuntu/django-main/django/conf/__init__.py", line 81 in __getattr__
  File "/home/ubuntu/django-main/tests/deprecation/test_storages.py", line 128 in test_access_warning
  File "/home/ubuntu/cpython-main/Lib/unittest/case.py", line 589 in _callTestMethod
  File "/home/ubuntu/cpython-main/Lib/unittest/case.py", line 634 in run
  File "/home/ubuntu/cpython-main/Lib/unittest/case.py", line 690 in __call__
  File "/home/ubuntu/django-main/django/test/testcases.py", line 292 in _setup_and_call
  File "/home/ubuntu/django-main/django/test/testcases.py", line 257 in __call__
  File "/home/ubuntu/cpython-main/Lib/unittest/suite.py", line 122 in run
  File "/home/ubuntu/cpython-main/Lib/unittest/suite.py", line 84 in __call__
  File "/home/ubuntu/cpython-main/Lib/unittest/runner.py", line 240 in run
  File "/home/ubuntu/django-main/django/test/runner.py", line 972 in run_suite
  File "/home/ubuntu/django-main/django/test/runner.py", line 1044 in run_tests
  File "/home/ubuntu/django-main/tests/./runtests.py", line 429 in django_tests
  File "/home/ubuntu/django-main/tests/./runtests.py", line 770 in <module>

Extension modules: pywatchman.bser, markupsafe._speedups (total: 2)
Aborted (core dumped)

sunmy2019 · 2023-04-06T17:51:30Z

I got a minimal repo!

class A:
    def __init__(self) -> None:
        self.bar = 0

    def __getattribute__(self, name):
        return super().__getattribute__(name)

    def __getattr__(self, name):
        if self.bar == 0:
            raise ValueError

    @property
    def foo(self):
        return self.__getattr__("foo")


A().foo

Running in older Python version, we got

Traceback (most recent call last):
  File "/home/ubuntu/django-main/tests/../a.py", line 17, in <module>
    A().foo
  File "/home/ubuntu/django-main/tests/../a.py", line 6, in __getattribute__
    return super().__getattribute__(name)
  File "/home/ubuntu/django-main/tests/../a.py", line 14, in foo
    return self.__getattr__("foo")
  File "/home/ubuntu/django-main/tests/../a.py", line 10, in __getattr__
    raise ValueError
ValueError

Call flow:

__getattribute__ called with name='foo'
property `foo` called

__getattribute__ called with name='__getattr__'
__getattr__ called with name='foo'
__getattribute__ called with name='bar'

But with the new python in main branch, (with out pydebug)

Exception ignored in tp_clear of: <class 'type'>
Traceback (most recent call last):
  File "/home/ubuntu/django-main/tests/../a.py", line 6, in __getattribute__
    return super().__getattribute__(name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
SystemError: <method-wrapper '__getattribute__' of A object at 0x7fc8af29e090> returned a result with an exception set
Traceback (most recent call last):
  File "/home/ubuntu/django-main/tests/../a.py", line 17, in <module>
    A().foo
  File "/home/ubuntu/django-main/tests/../a.py", line 9, in __getattr__
    if self.bar == 0:
       ^^^^^^^^
  File "/home/ubuntu/django-main/tests/../a.py", line 9, in __getattr__
    if self.bar == 0:
       ^^^^^^^^
  File "/home/ubuntu/django-main/tests/../a.py", line 9, in __getattr__
    if self.bar == 0:
       ^^^^^^^^
  [Previous line repeated 71 more times]
  File "/home/ubuntu/django-main/tests/../a.py", line 10, in __getattr__
    raise ValueError
ValueError

More precuriously, when I try to add some debug lines like this:

class A:
    def __init__(self) -> None:
        self.bar = 0

    def __getattribute__(self, name):
        return super().__getattribute__(name)

    def __getattr__(self, name):
        print(name)   ######### <----------- Here
        if self.bar == 0:
            raise ValueError

    @property
    def foo(self):
        return self.__getattr__("foo")


A().foo

The new python outputs,

foo
Traceback (most recent call last):
  File "/home/ubuntu/django-main/tests/../a.py", line 18, in <module>
    A().foo
  File "/home/ubuntu/django-main/tests/../a.py", line 9, in __getattr__
    print(name)
  File "/home/ubuntu/django-main/tests/../a.py", line 6, in __getattribute__
    return super().__getattribute__(name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/django-main/tests/../a.py", line 15, in foo
    return self.__getattr__("foo")
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/django-main/tests/../a.py", line 11, in __getattr__
    raise ValueError
ValueError

Everything just works fine? Looks like something in C is corrupting ❓ ❓ ❓

sunmy2019 · 2023-04-06T17:53:57Z

Now with pydebug on, I got the output from new python: (crashes with/without debug line)

foo
python: Python/ceval.c:821: _PyEval_EvalFrameDefault: Assertion `!_PyErr_Occurred(tstate)' failed.
Aborted (core dumped)

I can confirm, for this minimum repo, crash starts at aa0a73d

…ythonGH-103336) (cherry picked from commit 5d7d86f) Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>

(cherry picked from commit 5d7d86f) Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>

hauntsaninja · 2023-04-07T20:45:25Z

Thanks for the report and for narrowing this down!

The change that regressed this has been reverted and we've added a regression test, so I think this can be closed

…ython#103336)

zzzeek · 2023-05-24T19:46:22Z

WHEW, hi all, been bisecting all day (including getting better at bisecting large C code bases) to get here.

SQLAlchemy has been having failures on this in 3.12.0a7 and seem to be good in 3.12.0b1, just wanted to make sure this was logged.

FTR, here's our test case:

import sys


class Thing:
    @property
    def comparator(self):
        raise TypeError("a type error")

    def __getattr__(self, name):
        return getattr(self.comparator, name)

t1 = Thing()

try:
    t1.comparator
except TypeError:
    print("got typeerror as expected")

with the bug it throws a RecursionError

wangxiang-hz · 2023-06-01T07:34:21Z

3.12.0a7

I have confirmed that version 3.12.0a7 was released on April 4th and the bug does exist in this version. However, the bug has been fixed in version 3.12.0b1. Therefore, it is highly likely that your error is related to this issue.

sunmy2019 · 2023-06-01T11:35:38Z

your error is related to this issue

It is. Test coverage is added in #103272

felixxm added the type-bug An unexpected behavior, bug, or error label Apr 5, 2023

AlexWaygood added the 3.12 bugs and security fixes label Apr 5, 2023

Fidget-Spinner added the release-blocker label Apr 5, 2023

sunmy2019 mentioned this issue Apr 7, 2023

3.12.0a7 changes behavior of PropertyMock #103329

Closed

sobolevn mentioned this issue Apr 7, 2023

Revert "gh-102213: Optimize the performance of __getattr__ (GH-102248)" #103332

Merged

sunmy2019 added a commit to sunmy2019/cpython that referenced this issue Apr 7, 2023

add test case for pythongh-103272

d1f461f

sunmy2019 mentioned this issue Apr 7, 2023

gh-103272: add regression test for getattr that raises #103336

Merged

sobolevn added a commit to sunmy2019/cpython that referenced this issue Apr 7, 2023

Merge branch 'main' into test-case-for-pythongh-103272

21f7ec7

hauntsaninja pushed a commit that referenced this issue Apr 7, 2023

gh-103272: regression test for getattr exception in property (#103336)

5d7d86f

bedevere-bot mentioned this issue Apr 7, 2023

[3.11] gh-103272: regression test for getattr exception in property (GH-103336) #103351

Merged

miss-islington added a commit that referenced this issue Apr 7, 2023

gh-103272: regression test for getattr exception in property (GH-103336)

b8d1623

(cherry picked from commit 5d7d86f) Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>

hauntsaninja closed this as completed Apr 7, 2023

warsaw pushed a commit to warsaw/cpython that referenced this issue Apr 11, 2023

pythongh-103272: regression test for getattr exception in property (p…

a82048a

…ython#103336)

sunmy2019 mentioned this issue Apr 24, 2023

gh-102213: Optimizing the Performance of __getattr__ #102213 #102248

Merged

dimbleby mentioned this issue Apr 30, 2023

python 3.12 in pipelines python-poetry/poetry#7803

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Maximum recursion depth exceeded in getattr(). #103272

Maximum recursion depth exceeded in getattr(). #103272

felixxm commented Apr 5, 2023 •

edited by bedevere-bot

AlexWaygood commented Apr 5, 2023

Fidget-Spinner commented Apr 5, 2023

sunmy2019 commented Apr 6, 2023

sunmy2019 commented Apr 6, 2023

sunmy2019 commented Apr 6, 2023 •

edited

hauntsaninja commented Apr 7, 2023 •

edited

zzzeek commented May 24, 2023

wangxiang-hz commented Jun 1, 2023

sunmy2019 commented Jun 1, 2023

Maximum recursion depth exceeded in __getattr__(). #103272

Maximum recursion depth exceeded in __getattr__(). #103272

Comments

felixxm commented Apr 5, 2023 • edited by bedevere-bot

Bug report

Your environment

Linked PRs

AlexWaygood commented Apr 5, 2023

Fidget-Spinner commented Apr 5, 2023

sunmy2019 commented Apr 6, 2023

sunmy2019 commented Apr 6, 2023

sunmy2019 commented Apr 6, 2023 • edited

hauntsaninja commented Apr 7, 2023 • edited

zzzeek commented May 24, 2023

wangxiang-hz commented Jun 1, 2023

sunmy2019 commented Jun 1, 2023

Maximum recursion depth exceeded in getattr(). #103272

Maximum recursion depth exceeded in getattr(). #103272

felixxm commented Apr 5, 2023 •

edited by bedevere-bot

sunmy2019 commented Apr 6, 2023 •

edited

hauntsaninja commented Apr 7, 2023 •

edited