Python2.7: bad install directory or PYTHONPATH (does not support .pth files)

[enigmagroup]

This error happens when trying to build python2.7 with the most recent repo + package feeds:

( export PYTHONPATH="/home/x/openwrt-banana-mostrecent/staging_dir/target-arm_cortex-a8+vfpv3_uClibc-0.9.33.2_eabi/usr/lib/python2.7:/home/x/openwrt-banana-mostrecent/staging_dir/target-arm_cortex-a8+vfpv3_uClibc-0.9.33.2_eabi//usr/lib/python2.7/site-packages"; export PYTHONOPTIMIZE=""; export PYTHONDONTWRITEBYTECODE=1; cd /home/x/openwrt-banana-mostrecent/build_dir/host/Cython-0.20.2/; CFLAGS="-O2 -I/home/x/openwrt-banana-mostrecent/staging_dir/host/include -I/home/x/openwrt-banana-mostrecent/staging_dir/host/usr/include" CPPFLAGS="-I/home/x/openwrt-banana-mostrecent/staging_dir/host/include -I/home/x/openwrt-banana-mostrecent/staging_dir/host/usr/include" LDFLAGS="-L/home/x/openwrt-banana-mostrecent/staging_dir/host/lib -L/home/x/openwrt-banana-mostrecent/staging_dir/host/usr/lib" /home/x/openwrt-banana-mostrecent/staging_dir/host/bin/python2 ./setup.py install --no-cython-compile --prefix="/home/x/openwrt-banana-mostrecent/staging_dir/host/usr" ; )
running install
Checking .pth file support in /home/x/openwrt-banana-mostrecent/staging_dir/host/usr/lib/python2.7/site-packages/
/home/x/openwrt-banana-mostrecent/staging_dir/host/bin/python2 -E -c pass
TEST FAILED: /home/x/openwrt-banana-mostrecent/staging_dir/host/usr/lib/python2.7/site-packages/ does NOT support .pth files
error: bad install directory or PYTHONPATH

You are attempting to install a package to a directory that is not
on PYTHONPATH and which Python does not read ".pth" files from. The
installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:

/home/x/openwrt-banana-mostrecent/staging_dir/host/usr/lib/python2.7/site-packages/

and your PYTHONPATH environment variable currently contains:

'/home/x/openwrt-banana-mostrecent/staging_dir/target-arm_cortex-a8+vfpv3_uClibc-0.9.33.2_eabi/usr/lib/python2.7:/home/x/openwrt-banana-mostrecent/staging_dir/target-arm_cortex-a8+vfpv3_uClibc-0.9.33.2_eabi//usr/lib/python2.7/site-packages'

Here are some of your options for correcting the problem:

• You can choose a different installation directory, i.e., one that is
  on PYTHONPATH or supports .pth files

• You can add the installation directory to the PYTHONPATH environment
  variable. (It must then also be on PYTHONPATH whenever you run
  Python and want to use the package(s) you are installing.)

• You can set up the installation directory to support ".pth" files by
  using one of the approaches described here:

  https://pythonhosted.org/setuptools/easy_install.html#custom-installation-locations

Please make the appropriate changes for your system and try again.
make[3]: *** [/home/x/openwrt-banana-mostrecent/build_dir/host/Cython-0.20.2/.built] Error 1
make[3]: Leaving directory /home/x/openwrt-banana-mostrecent/feeds/enigmabox/python-cython' make[2]: *** [package/feeds/enigmabox/python-cython/host/compile] Error 2 make[2]: Leaving directory/home/x/openwrt-banana-mostrecent'
make[1]: *** [/home/x/openwrt-banana-mostrecent/staging_dir/target-arm_cortex-a8+vfpv3_uClibc-0.9.33.2_eabi/stamp/.package_compile] Error 2
make[1]: Leaving directory `/home/x/openwrt-banana-mostrecent'
make: *** [world] Error 2

[sbyx]

@commodo

[commodo]

You need to override PYTHONPATH in your Makefile to
PYTHONPATH:=$(PKG_INSTALL_DIR)/$(PYTHON_PKG_DIR)

That .pth error is a build check that python-setuptools has.

You can check these files for example(s):

• https://github.com/openwrt/packages/blob/master/lang/python-setuptools/Makefile
• https://github.com/openwrt/packages/blob/master/lang/python-pip/Makefile

[commodo]

Actually, if I think about it, I could try to add it in python-package.mk and see if it works.
I don't remember if I tried that or not.

Will try it now.

[commodo]

I just looked a bit closer to the code

PYTHONPATH:=$(PKG_INSTALL_DIR)/$(PYTHON_PKG_DIR)
define Build/Compile
    $(INSTALL_DIR) $(PYTHONPATH)
    $(call Build/Compile/PyMod,,\
        install --prefix="$(PKG_INSTALL_DIR)/usr" \
    )
endef

I recall now that I had to create that dir, so that the Python distutils would be able to install the packages there (where the PYTHONPATH var points to).

[sbyx]

Shouldn't it by created once python itself is compiled?

[enigmagroup]

Thx for your replies. It would be the most convenient if we could create that dir via any Makefile so that no manual intervention is needed.

[commodo]

Well, that folder is per-package.
It's not Python's.

From @enigmagroup's output it's not Python that's failing, but rather Cython, which is something else.
We may want to package that officially [later] though, since it's a real performance booster for Python code.

@enigmagroup : could you show me the Makefile you're using ?

[enigmagroup]

umm, which Makefile exactly?

[commodo]

The Cython Makefile.
I think that's the build that's failing, and not Python.

[enigmagroup]

From the oldpackages feed: http://git.openwrt.org/?p=packages.git;a=blob;f=lang/cython/Makefile;h=0e02fdf2e5d469c783d3c8b1e88f6743ca1aa7b8;hb=HEAD

[commodo]

Well then.

Let's try to get this in the new packages.
Any objections ?

[enigmagroup]

Cython itself compiles:
make package/cython/compile V=99
[...]
touch /home/x/openwrt-banana-mostrecent/staging_dir/target-arm_cortex-a8+vfpv3_uClibc-0.9.33.2_eabi/stamp/.cython_installed

[sbyx]

I see just wondering (feel free to ignore), couldn't you set PYTHONPATH in python-support.mk and create the directory at the beginning of Build/Compile/PyMod?

[commodo]

@sbyx : yep, I could do that; was thinking of something like that now
I'm thinking of using cython as a base to try it out.

[enigmagroup]

Uh oh. I just found out that it's not "cython" that fails to compile, but "python-cython" - which is a package I had created in my repo. https://github.com/enigmagroup/enigmabox-openwrt/blob/master/python-cython/Makefile

Sorry for the noise! So now all I have to do is to create that directory in my Makefile? I'll try it out right now.

[enigmagroup]

Ok, so far, it compiles. I found a Makefile that works, from an OpenWRT repo that has also a lot of Python3 stuff in it (https://gitorious.org/openwrt-cybjit/python3/commit/8eaacb12797787f4ffd629f8353a56d37b2613f0)

The reason I created a separate python-cython was because I needed a fair recent version of cython, 0.20.2 at the time of creation. I guess it was a dependency for beanstalkc or django.