New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

distutils bundled with python2.4 doesn't understand "python setup.py get_metadata" #664

Open
RichardBarrell opened this Issue Mar 21, 2014 · 11 comments

Comments

Projects
None yet
3 participants
@RichardBarrell

RichardBarrell commented Mar 21, 2014

I'm able to successfully package python2.4 with -s dir.

I'm able to successfully package eggs for it with -s dir, too, with some work.

I can't package python2.4 eggs with fpm's built in -s python support, though.

When I attempt to do so, I get a failure like:

fpm                                                                \
     -s python -t deb                                              \
     -n my-py24-setuptools                                         \
     -v ${MY_PY24_setuptools_VERSION}                              \
     -d 'my-py24'                                                  \
     --python-bin /opt/my-py24/bin/python                          \
     --no-python-fix-name                                          \
    --python-install-bin /opt/my-py24/bin                          \
    --python-install-lib /opt/my-py24/lib/python2.4/site-packages  \
    --description "setuptools 0.6c11 just for me"                  \
    /tmp/setuptools-0.6c11

Process failed: /bin/bash failed (exit code 1). Full command was:["/bin/bash", "-c", "env PYTHONPATH=/var/lib/gems/1.8/gems/fpm-1.0.2/lib/fpm/package /opt/my-py24/bin/python setup.py --command-packages=pyfpm get_metadata --output=/tmp/package-python-build20140321-17463-chpxhy/metadata.json"] {:level=>:error}
@RichardBarrell

This comment has been minimized.

RichardBarrell commented Mar 21, 2014

I've written a Vagrantfile and shell scripts for reproducing this on my machine, which I've checked into github at https://github.com/RichardBarrell/fpm-664-repro

@r4um r4um added the python label Mar 22, 2014

@jordansissel

This comment has been minimized.

Owner

jordansissel commented Apr 23, 2014

python 2.4 is 10 years old :(

@jordansissel

This comment has been minimized.

Owner

jordansissel commented Apr 23, 2014

Nevertheless, I'll try your vagrantfile and see what I can do :)

@jordansissel

This comment has been minimized.

Owner

jordansissel commented Apr 23, 2014

It looks like python 2.4 or setuptools 0.6c11 doesn't support adding custom commands to setup.py

@jordansissel

This comment has been minimized.

Owner

jordansissel commented Apr 23, 2014

Hmm... Quoting https://docs.python.org/2/distutils/extending.html

Beginning with Python 2.4, a third option is available, intended to allow new commands to be added which can support existing setup.py scripts without requiring modifications to the Python installation. This is expected to allow third-party extensions to provide support for additional packaging systems, but the commands can be used for anything distutils commands can be used for. A new configuration option, command_packages (command-line option --command-packages), can be used to specify additional packages to be searched for modules implementing commands.

@jordansissel

This comment has been minimized.

Owner

jordansissel commented Apr 23, 2014

So I wonder... I feel like python 2.4's distutils works correctly, but the problem is this external setuptools v 0.6c11 that is not functioning correctly?

Specifically, setuptools seems to use itself in the setup.py instead of using distutils. I don't really know how to compensate here. Best I can tell, the bug is in setuptools, not in python 2.4 or fpm.

@jordansissel

This comment has been minimized.

Owner

jordansissel commented Apr 23, 2014

python 2.6.5 works:

vagrant@lucid64:/tmp/setuptools-0.6c11$ env PYTHONPATH=/var/lib/gems/1.8/gems/fpm-1.0.2/lib/fpm/package python /tmp/setuptool*/setup.py --command-packages=pyfpm get_metadata --output=/tmp/package-python-build20140423-23740-dvimzd/metadata.json
running get_metadata

Python 2.4.6 does not.

vagrant@lucid64:/tmp/setuptools-0.6c11$ env PYTHONPATH=/var/lib/gems/1.8/gems/fpm-1.0.2/lib/fpm/package /opt/my-py24/bin/python /tmp/setuptool*/setup.py --command-packages=pyfpm get_metadata --output=/tmp/package-python-build20140423-23740-dvimzd/metadata.json
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: invalid command 'get_metadata'
@jordansissel

This comment has been minimized.

Owner

jordansissel commented Apr 23, 2014

strace shows python 2.4 actually loading pyfpm/get_metadata.py. python2.4 however complains it is missing pkg_resources. If I trick it into loading pkg_resources from this weird edge case of missing setuptools, it fails to load simplejson.

So basically, there's a chicken-and-egg problem here. You need to install setuptools and simplejson in your python 2.4 installation before you can use fpm with it. After that, you can use fpm to package setuptools!

@jordansissel

This comment has been minimized.

Owner

jordansissel commented Apr 23, 2014

Given all of the above, in order to use fpm to package python things you need a functional python environment that includes the following already installed:

  • python
  • simplejson
  • setuptools

We should be able to improve the error messages from fpm to let you know what to do

@jordansissel

This comment has been minimized.

Owner

jordansissel commented Apr 23, 2014

New error message based on this discussion:

root@lucid64:/vagrant# sh build_my_py24_setuptools.sh
Your python environment is missing json support (either json or simplejson python module). I cannot continue without this. {:error=>#<FPM::Util::ProcessFailed: /bin/bash failed (exit code 1). Full command was:["/bin/bash", "-c", "/opt/my-py24/bin/python -c 'import simplejson'"]>, :python=>"/opt/my-py24/bin/python", :level=>:error}
Process failed: Python (/opt/my-py24/bin/python) is missing simplejson or json modules. {:level=>:error}

jordansissel added a commit that referenced this issue Apr 23, 2014

Fix error checking for python json & pkg_resources
I forgot safesystem() raises an exception when it fails.

Now, if users have a python that is missing a feature we need in order
to have fpm build a package, you'll get an error like this:

      Your python environment is missing json support (either json or simplejson python module). I cannot continue without this.  {:error=>#<FPM::Util::ProcessFailed: /bin/bash failed (exit code 1).  Full command was:["/bin/bash", "-c", "/opt/my-py24/bin/python -c 'import simplejson'"]>, :python=>"/opt/my-py24/bin/python", :level=>:error}
      Process failed: Python (/opt/my-py24/bin/python) is missing simplejson or json modules. {:level=>:error}

This is for #664
@RichardBarrell

This comment has been minimized.

RichardBarrell commented Apr 24, 2014

To be honest, I don't really expect fpm to support this & I'd actually be pretty sad about it if attempting to support this caused fpm's Python egg handling code to get more complicated and possibly let bugs creep in. I'm happy to continue using -s dir, given that I'm using such a silly old version of Python.

prof-milki pushed a commit to prof-milki/xpm that referenced this issue Dec 18, 2014

jls
Fix error checking for python json & pkg_resources
I forgot safesystem() raises an exception when it fails.

Now, if users have a python that is missing a feature we need in order
to have fpm build a package, you'll get an error like this:

      Your python environment is missing json support (either json or simplejson python module). I cannot continue without this.  {:error=>#<FPM::Util::ProcessFailed: /bin/bash failed (exit code 1).  Full command was:["/bin/bash", "-c", "/opt/my-py24/bin/python -c 'import simplejson'"]>, :python=>"/opt/my-py24/bin/python", :level=>:error}
      Process failed: Python (/opt/my-py24/bin/python) is missing simplejson or json modules. {:level=>:error}

This is for jordansissel#664

prof-milki pushed a commit to prof-milki/xpm that referenced this issue Dec 27, 2014

Fix error checking for python json & pkg_resources
I forgot safesystem() raises an exception when it fails.

Now, if users have a python that is missing a feature we need in order
to have fpm build a package, you'll get an error like this:

      Your python environment is missing json support (either json or simplejson python module). I cannot continue without this.  {:error=>#<FPM::Util::ProcessFailed: /bin/bash failed (exit code 1).  Full command was:["/bin/bash", "-c", "/opt/my-py24/bin/python -c 'import simplejson'"]>, :python=>"/opt/my-py24/bin/python", :level=>:error}
      Process failed: Python (/opt/my-py24/bin/python) is missing simplejson or json modules. {:level=>:error}

This is for jordansissel#664

jordansissel added a commit that referenced this issue Apr 24, 2015

Fix error checking for python json & pkg_resources
I forgot safesystem() raises an exception when it fails.

Now, if users have a python that is missing a feature we need in order
to have fpm build a package, you'll get an error like this:

      Your python environment is missing json support (either json or simplejson python module). I cannot continue without this.  {:error=>#<FPM::Util::ProcessFailed: /bin/bash failed (exit code 1).  Full command was:["/bin/bash", "-c", "/opt/my-py24/bin/python -c 'import simplejson'"]>, :python=>"/opt/my-py24/bin/python", :level=>:error}
      Process failed: Python (/opt/my-py24/bin/python) is missing simplejson or json modules. {:level=>:error}

This is for #664

jordansissel added a commit that referenced this issue Jun 20, 2016

Fix error checking for python json & pkg_resources
I forgot safesystem() raises an exception when it fails.

Now, if users have a python that is missing a feature we need in order
to have fpm build a package, you'll get an error like this:

      Your python environment is missing json support (either json or simplejson python module). I cannot continue without this.  {:error=>#<FPM::Util::ProcessFailed: /bin/bash failed (exit code 1).  Full command was:["/bin/bash", "-c", "/opt/my-py24/bin/python -c 'import simplejson'"]>, :python=>"/opt/my-py24/bin/python", :level=>:error}
      Process failed: Python (/opt/my-py24/bin/python) is missing simplejson or json modules. {:level=>:error}

This is for #664
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment