Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

RecursionError: maximum recursion depth exceeded while calling a Python object #21

Closed
mikebonnet opened this issue Aug 27, 2019 · 21 comments
Closed

RecursionError: maximum recursion depth exceeded while calling a Python object #21

mikebonnet opened this issue Aug 27, 2019 · 21 comments
Assignees
@phdru
Labels
bug

Comments

@mikebonnet
Copy link
Contributor

When running the tests under Python 3.8 (3.8.0b3 in a Fedora Rawhide chroot) I'm getting recursion errors in the CheetahWrapper tests. It's reproduceable outside the tests with a simple cheetah fill command:

[mockbuild@eb00021cc65144128240dd7e5148c7fd tmpztxspttp]$ cat a.tmpl 
Hello, world!
[mockbuild@eb00021cc65144128240dd7e5148c7fd tmpztxspttp]$ cheetah fill a.tmpl 
Filling a.tmpl -> a.html
Traceback (most recent call last):
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/bin/cheetah", line 3, in <module>
    _cheetah()
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/CheetahWrapper.py", line 655, in _cheetah
    CheetahWrapper().main()
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/CheetahWrapper.py", line 147, in main
    meth()
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/CheetahWrapper.py", line 263, in fill
    self._compileOrFill()
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/CheetahWrapper.py", line 409, in _compileOrFill
    self._compileOrFillBundle(b)
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/CheetahWrapper.py", line 622, in _compileOrFillBundle
    tclass = TemplateClass.compile(file=src,
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/Template.py", line 767, in compile
    compiler = compilerClass(source, file,
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/Compiler.py", line 1659, in __init__
    f = open(file, 'r')
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/ImportManager.py", line 462, in importHook
    mod = _self_doimport(nm, ctx, fqname)
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/ImportManager.py", line 523, in doimport
    mod = director.getmod(nm)
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/ImportManager.py", line 363, in getmod
    mod = owner.getmod(nm)
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/ImportHooks.py", line 57, in getmod
    mod = DirOwner.getmod(self, name)
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/ImportManager.py", line 213, in getmod
    with open(py[0], 'r') as py_code_file:
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/ImportManager.py", line 462, in importHook
    mod = _self_doimport(nm, ctx, fqname)
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/ImportManager.py", line 523, in doimport
    mod = director.getmod(nm)
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/ImportManager.py", line 363, in getmod
    mod = owner.getmod(nm)
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/ImportHooks.py", line 57, in getmod
    mod = DirOwner.getmod(self, name)
...
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/ImportHooks.py", line 57, in getmod
    mod = DirOwner.getmod(self, name)
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/ImportManager.py", line 213, in getmod
    with open(py[0], 'r') as py_code_file:
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/ImportManager.py", line 462, in importHook
    mod = _self_doimport(nm, ctx, fqname)
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/ImportManager.py", line 523, in doimport
    mod = director.getmod(nm)
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/ImportManager.py", line 363, in getmod
    mod = owner.getmod(nm)
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/ImportHooks.py", line 57, in getmod
    mod = DirOwner.getmod(self, name)
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/ImportManager.py", line 213, in getmod
    with open(py[0], 'r') as py_code_file:
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/ImportManager.py", line 441, in importHook
    pkgnm = packageName(importernm)
  File "/builddir/build/BUILDROOT/python-cheetah-3.2.3-1.fc32.x86_64/usr/lib64/python3.8/site-packages/Cheetah/ImportManager.py", line 108, in packageName
    for i in range(len(s) - 1, -1, -1):
RecursionError: maximum recursion depth exceeded while calling a Python object

This is using the 3.2.3 release of Cheetah from pypi.
@phdru
Copy link
Member

phdru commented Aug 27, 2019

Works for me with Python 3.8.0b3 (Debian 9 stretch, temporary virtual environment):

$ mktmpenv -p python3.8
Running virtualenv with interpreter /home/phd/.local/bin/python3.8
(tmp-ae8c1d7c52dcdc) tmp-ae8c1d7c52dcdc $ pip install Cheetah3Collecting Cheetah3
  Downloading https://files.pythonhosted.org/packages/3e/16/c711180492c9f40fb64dffb436fe1b91e3031637b478edb8de3c4b74097a/Cheetah3-3.2.3.tar.gz (875kB)
     |################################| 880kB 2.1MB/s 
Building wheels for collected packages: Cheetah3
  Building wheel for Cheetah3 (setup.py) ... done
  Created wheel for Cheetah3: filename=Cheetah3-3.2.3-cp38-cp38-linux_i686.whl size=182386 sha256=2ee27beb04809bd80fc036008a518621e95d6d9b04b0380e7ab7a464c3eb6bc0
  Stored in directory: /home/phd/.cache/pip/wheels/33/20/34/73e8bfd45981dfef3edad9b301c5498a86b4cef496a7b29555
Successfully built Cheetah3
Installing collected packages: Cheetah3
Successfully installed Cheetah3-3.2.3
(tmp-ae8c1d7c52dcdc) tmp-ae8c1d7c52dcdc $ cd ~/tmp/
(tmp-ae8c1d7c52dcdc) tmp $ cheetah fill a.tmpl 
Filling a.tmpl -> a.html
(tmp-ae8c1d7c52dcdc) tmp $ cat a.html 
Hello, world!

How can I reproduce the problem?

@mikebonnet
Copy link
Contributor Author

You can reproduce using the Fedora 32 docker image.

docker run -it fedora:32 /bin/bash
curl https://files.pythonhosted.org/packages/3e/16/c711180492c9f40fb64dffb436fe1b91e3031637b478edb8de3c4b74097a/Cheetah3-3.2.3.tar.gz | tar xz
echo "Hello, world!" > a.tmpl
PYTHONPATH=Cheetah3-3.2.3 python3 Cheetah3-3.2.3/bin/cheetah fill a.tmpl

@phdru
Copy link
Member

phdru commented Aug 27, 2019

All my Linux boxes currently run 32-bit Debian so I cannot use Docker yet. I have plans to upgrade to 64-bit but it'd take time.

Why do you use curl instead of pip install?

@phdru
Copy link
Member

phdru commented Aug 27, 2019

I found a very old CentOS-based installation in VirtualBox. I only have Python 2.7 there. Your example passed fine.

I'll try to install Fedora into a new virtual machine but that could take a lot of time.

@mikebonnet
Copy link
Contributor Author

Docker is also installable on Mac and Windows, if you have access to those environments.

@phdru
Copy link
Member

phdru commented Aug 28, 2019

With VirtualBox I can install Fedora directly. Or I can install 64-bit Debian, install Docker into it and install Fedora into Docker. :-)

@phdru
Copy link
Member

phdru commented Aug 28, 2019

Is the problem manifested only with Python 3.8 or any other Python version?

@mikebonnet
Copy link
Contributor Author

Hmmm, no, I can also reproduce it under Python 3.7.4 (Fedora 30). Interestingly, I can't reproduce it on my local machine, which is running Fedora 30, but I can reproduce it in a clean chroot or in a Docker container. It may be influenced by other Python libraries being installed at the same time?

@phdru
Copy link
Member

phdru commented Aug 28, 2019 •
edited

It may be influenced by other Python libraries being installed at the same time?

Seems so. Though I though it shouldn't behave that way.

Meanwhile I failed to install Fedora 30 64-bit into VirtualBox — installer halts almost at the beginning. Trying to install Debian 10 64-bit (for Docker)…

Upd. Debian installed is a bit more clever — it reported I don't have enough memory. So I recreated a virtual machine for Fedora with more memory. Anaconda just started.

@phdru
Copy link
Member

phdru commented Sep 1, 2019

I installed Fedora 30 64bit. Now I'm collecting the list of libraries and headers needed to compile Python. OpenSSL, ncurses, terminfo, readline, zlib and more…

@mikebonnet
Copy link
Contributor Author

@phdru Are you able to reproduce the problem with the python that comes with Fedora 30?

@phdru
Copy link
Member

phdru commented Sep 3, 2019

I didn't try. Should I? Is the problem reproducible for you? With what version — Python 2 or Python 3?

Meanwhile I managed to install development tools, libraries and headers. I compiled Python 3.8.0b3 but also didn't try Cheetah yet. I'm slowly moving, not having enough spare time. But I'm moving…

@mikebonnet
Copy link
Contributor Author

@phdru Yes, it's reproduceable for me in a clean Fedora 30 environment (Docker image or mock chroot) using Python 3.7.4. I'd be interested if you can reproduce it in a similar environment.

@phdru
Copy link
Member

phdru commented Sep 4, 2019

With default python3:

$ python3.7
Python 3.7.4 (default, Jul  9 2019, 16:32:37) 
[GCC 9.1.1 20190503 (Red Hat 9.1.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
$ mkvirtualenv -p python3.7 test-cheetah
Running virtualenv with interpreter /usr/bin/python3.7
Already using interpreter /usr/bin/python3.7
Using base prefix '/usr'
  No LICENSE.txt / LICENSE found in source
New python executable in /home/phd/.virtualenvs/test-cheetah/bin/python3.7
Also creating executable in /home/phd/.virtualenvs/test-cheetah/bin/python
Installing setuptools, pip, wheel...
done.
virtualenvwrapper.user_scripts creating /home/phd/.virtualenvs/test-cheetah/bin/predeactivate
virtualenvwrapper.user_scripts creating /home/phd/.virtualenvs/test-cheetah/bin/postdeactivate
virtualenvwrapper.user_scripts creating /home/phd/.virtualenvs/test-cheetah/bin/preactivate
virtualenvwrapper.user_scripts creating /home/phd/.virtualenvs/test-cheetah/bin/postactivate
virtualenvwrapper.user_scripts creating /home/phd/.virtualenvs/test-cheetah/bin/get_env_details
(test-cheetah) $ type -a pip
pip is /home/phd/.virtualenvs/test-cheetah/bin/pip
pip is /usr/bin/pip
pip is /bin/pip
(test-cheetah) $ pip install cheetah3
Collecting cheetah3
  Downloading https://files.pythonhosted.org/packages/8c/39/13d33f2bb40a132c069696c16324e3c7eed2f301a23f026ad519f0289c22/Cheetah3-3.2.3-cp37-cp37m-manylinux1_x86_64.whl (669kB)
     |################################| 675kB 1.7MB/s 
Installing collected packages: cheetah3
Successfully installed cheetah3-3.2.3
(test-cheetah) $ echo 'Hello, world!' >a.tmpl
(test-cheetah) $ cheetah fill a.tmpl 
Filling a.tmpl -> a.html
(test-cheetah) $ cat a.html 
Hello, world!

No problem.

@phdru
Copy link
Member

phdru commented Sep 4, 2019 

$ python3.8
Python 3.8.0b3 (default, Sep  4 2019, 18:27:11) 
[GCC 9.2.1 20190827 (Red Hat 9.2.1-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> 
$ mkvirtualenv -p python3.8 test-cheetah
Running virtualenv with interpreter /home/phd/.local/bin/python3.8
Already using interpreter /home/phd/.local/bin/python3.8
Using base prefix '/home/phd/.local'
New python executable in /home/phd/.virtualenvs/test-cheetah/bin/python3.8
Also creating executable in /home/phd/.virtualenvs/test-cheetah/bin/python
Installing setuptools, pip, wheel...
done.
virtualenvwrapper.user_scripts creating /home/phd/.virtualenvs/test-cheetah/bin/predeactivate
virtualenvwrapper.user_scripts creating /home/phd/.virtualenvs/test-cheetah/bin/postdeactivate
virtualenvwrapper.user_scripts creating /home/phd/.virtualenvs/test-cheetah/bin/preactivate
virtualenvwrapper.user_scripts creating /home/phd/.virtualenvs/test-cheetah/bin/postactivate
virtualenvwrapper.user_scripts creating /home/phd/.virtualenvs/test-cheetah/bin/get_env_details
(test-cheetah) $ pip install cheetah3
Collecting cheetah3
  Downloading https://files.pythonhosted.org/packages/3e/16/c711180492c9f40fb64dffb436fe1b91e3031637b478edb8de3c4b74097a/Cheetah3-3.2.3.tar.gz (875kB)
     |################################| 880kB 1.7MB/s 
Building wheels for collected packages: cheetah3
  Building wheel for cheetah3 (setup.py) ... done
  Created wheel for cheetah3: filename=Cheetah3-3.2.3-cp38-cp38-linux_x86_64.whl size=181500 sha256=ccb2b75f12838f71542582561792c3a42504e0af165b30539158b84b533717b4
  Stored in directory: /home/phd/.cache/pip/wheels/33/20/34/73e8bfd45981dfef3edad9b301c5498a86b4cef496a7b29555
Successfully built cheetah3
Installing collected packages: cheetah3
Successfully installed cheetah3-3.2.3
(test-cheetah) $ cheetah fill a.tmpl 
Filling a.tmpl -> a.html
(test-cheetah) $ cat a.html 
Hello, world!

@phdru
Copy link
Member

phdru commented Sep 5, 2019 •
edited

You can reproduce using the Fedora 32 docker image.

docker run -it fedora:32 /bin/bash
curl https://files.pythonhosted.org/packages/3e/16/c711180492c9f40fb64dffb436fe1b91e3031637b478edb8de3c4b74097a/Cheetah3-3.2.3.tar.gz | tar xz
echo "Hello, world!" > a.tmpl
PYTHONPATH=Cheetah3-3.2.3 python3 Cheetah3-3.2.3/bin/cheetah fill a.tmpl

With docker (installed into 64-bit virtual machine running on a 32-bit host) I finally reproduced the problem. Investigating…

Upd. Also reproducible with image fedora:30 and pip3 install Cheetah3.

@phdru
Copy link
Member

phdru commented Sep 5, 2019 •
edited

Quick workaround:

cheetah fill a.tmpl >/dev/null
cheetah fill a.tmpl >logfile
cheetah fill a.tmpl | ${PAGER:-more}

Redirect stdout.

@phdru
Copy link
Member

phdru commented Sep 11, 2019 •
edited

I found the culprit. ImportManager calls open() which tries to import module _bootlocale.py which triggers ImportManager. Infinite recursion.

If you're in a hurry the temporary fix is simple: add import _bootlocale at the beginning of ImportManager.py.

Meanwhile I found a reproducible test case fe0ff3b. Fails: https://travis-ci.org/CheetahTemplate3/cheetah3/builds/583806358. But now I can continue debugging without any virtual machines or containers.

@phdru
Copy link
Member

phdru commented Sep 14, 2019

Can you try the fix d9241f8? It's now at the head of master so just try

pip install 'git+https://github.com/CheetahTemplate3/cheetah3.git#egg=cheetah3'

@mikebonnet
Copy link
Contributor Author

Yes, this works for me, nice!

@phdru phdru added the bug label Sep 17, 2019
@phdru phdru self-assigned this Sep 17, 2019
@phdru
Copy link
Member

phdru commented Sep 17, 2019

Thank you for reporting!

@phdru phdru closed this as completed Sep 17, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@phdru phdru

Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@phdru @mikebonnet