Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Creating venv fails #21

Closed
tgeulig opened this issue Feb 18, 2020 · 7 comments
Closed

Creating venv fails #21

tgeulig opened this issue Feb 18, 2020 · 7 comments

Comments

@tgeulig
Copy link

tgeulig commented Feb 18, 2020

I can't even create the virtual environment and get "Failed to import site module".
This is crossenv 0.6 on opensuse leap 15.1. Target host is Raspberry Pi with a custom arm toolchain. Python on suse is 3.6.10 and 3.6.5 on target. Close enough?

/usr/bin/python3 -m crossenv /nentec/work/geulig/NENUX/nenux.git/user/python/build/python venv
WARNING: CC is a compound command (arm-pgc-linux-gnueabi-gcc -mlittle-endian -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/include -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/mach-bcm/include -I/nentec/work/geulig/NENUX/nenux.git/include -O1 -pipe -fno-common -fno-builtin -Wall  -Dlinux -D__linux__ -Dunix  -DEMBED -mlittle-endian -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/include -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/mach-bcm/include)
WARNING: This can cause issues for modules that don't expect it.
WARNING: Consider setting CC='arm-pgc-linux-gnueabi-gcc' and CFLAGS='-mlittle-endian -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/include -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/mach-bcm/include -I/nentec/work/geulig/NENUX/nenux.git/include -O1 -pipe -fno-common -fno-builtin -Wall -Dlinux -D__linux__ -Dunix -DEMBED -mlittle-endian -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/include -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/mach-bcm/include'
Failed to import the site module
Traceback (most recent call last):
  File "/usr/lib64/python3.6/sysconfig.py", line 145, in _subst_vars
    return s.format(**local_vars)
KeyError: 'platlibdir'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib64/python3.6/sysconfig.py", line 148, in _subst_vars
    return s.format(**os.environ)
KeyError: 'installed_base'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/nentec/work/geulig/NENUX/nenux.git/user/python_modules/venv/lib/site.py", line 62, in <module>
    stdlib = os.path.normpath(sysconfig.get_path('stdlib'))
  File "/usr/lib64/python3.6/sysconfig.py", line 512, in get_path
    return get_paths(scheme, vars, expand)[name]
  File "/usr/lib64/python3.6/sysconfig.py", line 502, in get_paths
    return _expand_vars(scheme, vars)
  File "/usr/lib64/python3.6/sysconfig.py", line 169, in _expand_vars
    res[key] = os.path.normpath(_subst_vars(value, vars))
  File "/usr/lib64/python3.6/sysconfig.py", line 150, in _subst_vars
    raise AttributeError('{%s}' % var)
AttributeError: {'installed_base'}
ERROR: Command '['/home/nentec/work/geulig/NENUX/nenux.git/user/python_modules/venv/cross/bin/python3', '-m', 'pip', '--disable-pip-version-check', 'install', '--ignore-installed', '--prefix=/home/nentec/work/geulig/NENUX/nenux.git/user/python_modules/venv/cross', 'pip==18.1', 'setuptools==40.6.2']' returned non-zero exit status 1.
make[2]: *** [Makefile:61: venv] Fehler 1
@benfogle
Copy link
Owner

It looks like you're using the system version of python. First, it's important that build-python and cross-python be the exact same version, 3.6.5 in your case. Second, the version of Python that came with the system has distro-specific patches can cause incompatibilities between build-python and host-python. I highly recommend that you build Python 3.6.5 for your workstation from the same source used to build the version on the Raspberry Pi, and run crossenv using that.

@tgeulig
Copy link
Author

tgeulig commented Feb 20, 2020

Meanwhile I've update host-python to 3.5.10 to be on the safe side, but that results in the same error. So I will now compile my own python 3.5.10 on host as well.

@tgeulig
Copy link
Author

tgeulig commented Feb 21, 2020

Build-python and host-python are now identical. Now I get this error:

[tgeulig@sw-dev](master)> PYTHONSTARTUP= PYTHONHOME=/opt/python-3.6.10 PYTHONPATH=/opt/python-3.6.10/lib64/python3.6/lib-dynload /opt/python-3.6.10/bin/python3 -m crossenv /nentec/work/geulig/NENUX/nenux.git/user/python/build/python venv
WARNING: CC is a compound command (arm-pgc-linux-gnueabi-gcc -mlittle-endian -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/include -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/mach-bcm/include -I/nentec/work/geulig/NENUX/nenux.git/include -O1 -pipe -fno-common -fno-builtin -Wall  -Dlinux -D__linux__ -Dunix  -DEMBED -mlittle-endian -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/include -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/mach-bcm/include)
WARNING: This can cause issues for modules that don't expect it.
WARNING: Consider setting CC='arm-pgc-linux-gnueabi-gcc' and CFLAGS='-mlittle-endian -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/include -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/mach-bcm/include -I/nentec/work/geulig/NENUX/nenux.git/include -O1 -pipe -fno-common -fno-builtin -Wall -Dlinux -D__linux__ -Dunix -DEMBED -mlittle-endian -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/include -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/mach-bcm/include'
WARNING: CXX is a compound command (arm-pgc-linux-gnueabi-g++ -mlittle-endian -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/include -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/mach-bcm/include -I/nentec/work/geulig/NENUX/nenux.git/include)
WARNING: This can cause issues for modules that don't expect it.
WARNING: Consider setting CXX='arm-pgc-linux-gnueabi-g++' and CXXFLAGS='-mlittle-endian -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/include -I/nentec/work/geulig/NENUX/nenux.git/linux-4.19.x/arch/arm/mach-bcm/include -I/nentec/work/geulig/NENUX/nenux.git/include'
ERROR: Command '['/home/nentec/work/geulig/NENUX/nenux.git/user/python_modules/venv/build/bin/python3', '-Im', 'ensurepip', '--upgrade', '--default-pip']' returned non-zero exit status 1.

If I run the last command directly, I get:

[tgeulig@sw-dev](master)> /home/nentec/work/geulig/NENUX/nenux.git/user/python_modules/venv/build/bin/python3 -Im ensurepip --upgrade --default-pip
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Traceback (most recent call last):
  File "/opt/python-3.6.10/lib/python3.6/runpy.py", line 183, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "/opt/python-3.6.10/lib/python3.6/runpy.py", line 142, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "/opt/python-3.6.10/lib/python3.6/runpy.py", line 109, in _get_module_details
    __import__(pkg_name)
  File "/opt/python-3.6.10/lib/python3.6/ensurepip/__init__.py", line 5, in <module>
    import tempfile
  File "/opt/python-3.6.10/lib/python3.6/tempfile.py", line 45, in <module>
    from random import Random as _Random
  File "/opt/python-3.6.10/lib/python3.6/random.py", line 42, in <module>
    from math import log as _log, exp as _exp, pi as _pi, e as _e, ceil as _ceil
ModuleNotFoundError: No module named 'math'

I'm not sure how build-python is supposed to find its modules here ...
(using crossenv 0.7 now BTW)

@benfogle
Copy link
Owner

It sounds like lib-dynload is missing somehow. It should have been installed to /opt/python-3.6.10/lib/python3.6/lib-dynload when you ran make install. Can you verify that it's there? What arguments did you use with configure/make/make install?

@tgeulig
Copy link
Author

tgeulig commented Mar 3, 2020

"/opt/python-3.6.10/lib/python3.6/lib-dynload/" isn't there, but "/opt/python-3.6.10/lib64/python3.6/lib-dynload/" is.

Complie:

./configure --prefix=/opt/python-3.6.10 --disable-shared --disable-ipv6 --enable-unicode=ucs4 --without-doc-strings
make
make install

@benfogle
Copy link
Owner

benfogle commented Mar 5, 2020

I'm unable to reproduce that when I build Python with those arguments in an openSUSE Leap Docker image. I get /opt/python-3.6.10/lib/python3.6/lib-dynload as expected. If I rename it to /opt/python-3.6.10/lib64/python3.6/lib-dynload, I get the same error you are seeing.

I don't know what's causing it to install incorrectly, but a quick fix:

ln -s /opt/python-3.6.10/lib64/python3.6/lib-dynload /opt/python-3.6.10/lib/python3.6/lib-dynload

@tgeulig
Copy link
Author

tgeulig commented Mar 6, 2020

Thank you for your help, with this symbolic link I am finally able to compile some modules.
I don't know, why the link is necessary, but as it seems to be an issue of my environment, I am closing this issue.

@tgeulig tgeulig closed this as completed Mar 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@benfogle @tgeulig