Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pip: No setuptools found in remote host, please install it first. #47361

Closed
varunchopra opened this issue Oct 19, 2018 · 22 comments · Fixed by #47743
Closed

pip: No setuptools found in remote host, please install it first. #47361

varunchopra opened this issue Oct 19, 2018 · 22 comments · Fixed by #47743
Labels
affects_2.8 bug module python3 support:core traceback

Comments

@varunchopra
Copy link
Contributor

@varunchopra varunchopra commented Oct 19, 2018

SUMMARY

pip seems to be broken in 2.8.

ISSUE TYPE
  • Bug Report
COMPONENT NAME

pip

ANSIBLE VERSION
ansible 2.8.0.dev0 (win-part-devel 202a9aa06c) last updated 2018/09/28 21:59:26 (GMT +000)
  config file = None
  configured module search path = [u'/home/vagrant/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
  ansible python module location = /workspace/ansible/lib/ansible
  executable location = /workspace/ansible/bin/ansible
  python version = 2.7.15rc1 (default, Apr 15 2018, 21:51:34) [GCC 7.3.0]
STEPS TO REPRODUCE

The following works in Ansible 2.6 without errors:

- name: Install dependencies
  apt:
    name:
      - python3
      - python3-pip
      - python3-setuptools
    state: latest
    update_cache: True
- name: Install required pip packages
  pip:
    name:
      - argparse
      - bleach
      - requests-toolbelt
    executable: pip3

But in Ansible 2.8:

fatal: []: FAILED! => {"changed": false, "msg": "No setuptools found in remote host, please install it first."}
EXPECTED RESULTS
TASK [Install required pip packages] ***********************************************************************
changed: [1]
changed: [2]
changed: [3]
ACTUAL RESULTS
The full traceback is:
WARNING: The below traceback may *not* be related to the actual failure.
  File "/tmp/ansible_pip_payload_mQPVtz/__main__.py", line 240, in <module>
    from pkg_resources import Requirement

fatal: [x]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "chdir": null,
            "editable": false,
            "executable": "pip3",
            "extra_args": null,
            "name": [
                "argparse",
                "bleach",
                "requests-toolbelt"
            ],
            "requirements": null,
            "state": "present",
            "umask": null,
            "use_mirrors": true,
            "version": null,
            "virtualenv": null,
            "virtualenv_command": "virtualenv",
            "virtualenv_python": null,
            "virtualenv_site_packages": false
        }
    },
    "msg": "No setuptools found in remote host, please install it first."
}
@ansibot
Copy link
Contributor

@ansibot ansibot commented Oct 19, 2018

Hi @chopraaa, thank you for submitting this issue!

click here for bot help

@ansibot
Copy link
Contributor

@ansibot ansibot commented Oct 19, 2018

Files identified in the description:

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

@ansibot
Copy link
Contributor

@ansibot ansibot commented Oct 19, 2018

@ansibot ansibot added affects_2.8 bug module needs_triage python3 support:core traceback labels Oct 19, 2018
@robinro
Copy link
Contributor

@robinro robinro commented Oct 19, 2018

cc @HD650

@ansibot ansibot removed the needs_triage label Oct 19, 2018
@richard-viney
Copy link

@richard-viney richard-viney commented Oct 20, 2018

Just encountered this after upgrading from 2.6.5 to 2.7.0 - maybe this isn't a 2.8 issue?

Is there a known workaround?

@varunchopra
Copy link
Contributor Author

@varunchopra varunchopra commented Oct 20, 2018

Like I said, it works on 2.6 but not on 2.8. Now we know it doesn't work on 2.7 either. :-)

As a workaround, you can remove this check:

if not HAS_SETUPTOOLS:
module.fail_json(msg="No setuptools found in remote host, please install it first.")

@richard-viney
Copy link

@richard-viney richard-viney commented Oct 21, 2018

Great, I seem to have worked around it by installing the python-pip package on the host (I'm on Ubuntu 18.04).

@HD650
Copy link
Contributor

@HD650 HD650 commented Oct 21, 2018

It‘s because the remote machine has no setuptools which is required by pip. You can use apt-get install -y python-setuptools first to install it and then use the pip module. Also like @richard-viney said, some version of pip package includes setuptools, so just install python-pip can solve this too.

@varunchopra
Copy link
Contributor Author

@varunchopra varunchopra commented Oct 21, 2018

@HD650 I need python2 setuptools for using python3?

@jborean93
Copy link
Contributor

@jborean93 jborean93 commented Oct 21, 2018

When running with -vvv you can see the Python that is being used to execute the module, here is an example for ping;

192.168.56.23> SSH: EXEC ssh -C -o ControlMaster=auto -o ControlPersist=60s -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o User=ansible -o ConnectTimeout=10 -o ControlPath=/Users/jborean/.ansible/cp/2b544c5737 -tt 192.168.56.23 '/bin/sh -c '"'"'/usr/bin/python /home/ansible/.ansible/tmp/ansible-tmp-1540156197.54-175245074395330/AnsiballZ_ping.py && sleep 0'"'"''

We can see it executing /usr/bin/python and this would be where setuptools need to be installed. If you wish to use a different Python you can set the interpreter with /usr/bin/python.

@varunchopra
Copy link
Contributor Author

@varunchopra varunchopra commented Oct 21, 2018

I don't understand why. I have python-simplejson installed on the managed host, and I'm using Ansible to install python3 and whatever relevant packages I need installed.

  • This worked earlier, on 2.6, and with good reason.
  • I shouldn't need to install python-setuptools if I won't be using Python for anything other than running Ansible. (Note: python-simplejson being already present)
  • I shouldn't need to use Python 3 as the interpreter since in most cases for me it isn't present on the server to begin with. I'm also specifying the executable as pip3.

This can be closed if intentional. I don't mean to step on anyone's toes.

@HD650
Copy link
Contributor

@HD650 HD650 commented Oct 21, 2018

@chopraaa I think the problem is:

  1. you have a python2 in the remote machine, and Ansible is running in python2
  2. you use pip module in the remote machine, now this pip module is running in python2
  3. I added a new feature in pip module to support version specifier in Ansible 2.7, this feature requires setuptools to parse the version specifier
  4. So I think you need to install setuptools for python2 in this case since python2 is responsible for executing Ansible, please try install python-setuptools rather than python3-setuptools

oharsta added a commit to OpenConext/OpenConext-deploy that referenced this issue Oct 22, 2018
Nani-o added a commit to Nani-o/ansible-role-django that referenced this issue Oct 23, 2018
@funkyfuture
Copy link

@funkyfuture funkyfuture commented Oct 25, 2018

afaict, there is a design bug that causes this. the pip module code checks whether setuptools is available in the site packages that itself is using. that may be false, even though the interpreter selected with virtualenv_python does have it in its site-packages.

i cannot comprehend why @HD650 added this with 501503f, it doesn't seem to be functionally related to the feature the commit provides.

@funkyfuture
Copy link

@funkyfuture funkyfuture commented Oct 26, 2018

sorry, i should have properly read the post above that explains why setuptools is needed.

@twekberg
Copy link

@twekberg twekberg commented Oct 26, 2018

I'm running ansible 2.7.0, using python3, Ubuntu 16.04, and am having the same problem. I have python3-setuptools installed . Tried to update pip with this:

    pip: virtualenv="{{ VENV }}" name=pip state=latest

(where VENV points to where my virtual env is stored) and got the error complaining about setuptools not being found:

fatal: [new-tracker]: FAILED! => {"changed": false, "msg": "No setuptools found in remote host, please install it first."}

The pip module should pick pip3 since in my venv python -V is 3.5.2. You would think it should be looking for python3-setuptools. I ended up using shell:

    shell: source {{ VENV }}/bin/activate && pip3 install -U pip
    args:
      executable: /bin/bash

@jobevers
Copy link
Contributor

@jobevers jobevers commented Oct 26, 2018

I got the same error, so replaced the pip module with the command module and it works just fine:

before:

     - name: Install python dependencies
       pip:
        requirements: /build/requirements.txt
        executable: /usr/local/bin/pip

after:

     - name: Install python dependencies
       command: /usr/local/bin/pip install -r /build/requirements.txt

It seems to me that the setuptool check is either flawed or unnecessary.

@mkjpryor-stfc
Copy link

@mkjpryor-stfc mkjpryor-stfc commented Oct 29, 2018

I am having this same problem when using virtualenv and virtualenv_command.

It seems that the pip module is making the check before creating the virtualenv, because if it did the check on the virtualenv after creating it, it would see that setuptools and pip are both present in the virtualenv. I don't really want to install them into my system Python unnecessarily.

2.6 works, 2.7+ doesn't.

@mkjpryor-stfc
Copy link

@mkjpryor-stfc mkjpryor-stfc commented Oct 29, 2018

The problem is that HAS_SETUPTOOLS in the check is based on the Python that Ansible is running in (the system Python), whereas I want to use pip in a Python 3 virtual environment.

Again, this worked prior to 2.7.

@mkjpryor-stfc
Copy link

@mkjpryor-stfc mkjpryor-stfc commented Oct 29, 2018

One way to solve this would be to check the exit code of python -c "import setuptools" on the Python that the pip command is actually going to be using, after any virtualenv creation has been done.

@varunchopra
Copy link
Contributor Author

@varunchopra varunchopra commented Oct 29, 2018

@mkjpryor-stfc This was botched modified in 501503f where Ansible requires setuptools to be present for the interpreter.

I think this is done to check which packages are installed now. Hopefully they can update the documentation the next time they introduce a breaking change.

@ansible ansible locked and limited conversation to collaborators Oct 29, 2018
@jborean93
Copy link
Contributor

@jborean93 jborean93 commented Oct 29, 2018

I've locked the conversation as the damage has been done and setuptools is now a requirement for the pip module. I'm sure there can be a future feature that only uses setuptools if a version specifier is set but for now I'll settle with having a more descriptive error of what's happening. The PR #47743 will improve the error message when setuptools is not installed as well as add the entry to the 2.7 porting guide.

Just to clarify the situation, setuptools is required by the pip module and must be installed in the Python that is executing the module. This may or may not be same Python as the one specified by executable or virtualenv_python. Typically Python at /usr/bin/python will execute modules unless you are running on implicit localhost or have set ansible_python_interpreter.

@webknjaz
Copy link
Member

@webknjaz webknjaz commented Oct 31, 2018

@jborean93 actually, there are also complications in how ansible parses module arguments, so we needed to reconstruct actual package specs and to checks on the module side. I believe it's not possible to have such a feature (unless we vendor things).

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
affects_2.8 bug module python3 support:core traceback
Projects
None yet
Development

Successfully merging a pull request may close this issue.