Ansible formula modifies PYTHONPATH and causes Python scripts within playbooks to fail

[bkeroackdsc]

The Ansible formula appears to modify PYTHONPATH internally to vendored versions of some modules. This causes Python scripts executed within Ansilbe playbooks to potentially fail if there are module conflicts.

Test Ansible playbook tasks:

    - name: Getting PYTHONPATH
      local_action: shell python3 -c 'import sys; print(":".join(sys.path))'
      register: pythonpath

    - debug:
        msg: "PYTHONPATH: {{ pythonpath.stdout }}"

Output:

TASK: [Getting PYTHONPATH] ****************************************************
changed: [localhost -> 127.0.0.1]

TASK: [debug ] ****************************************************************
ok: [localhost] => {
    "msg": "PYTHONPATH: :/usr/local/Cellar/ansible/1.8.2/libexec/lib/python2.7/site-packages:/usr/local/Cellar/ansible/1.8.2/libexec/vendor/lib/python2.7/site-packages:/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python34.zip:/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4:/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/plat-darwin:/usr/local/Cellar/python3/3.4.2_1/Frameworks/Python.framework/Versions/3.4/lib/python3.4/lib-dynload:/usr/local/lib/python3.4/site-packages"
}

Note the first two entries:

• /usr/local/Cellar/ansible/1.8.2/libexec/lib/python2.7/site-packages
• /usr/local/Cellar/ansible/1.8.2/libexec/vendor/lib/python2.7/site-packages

These entries alter the Python execution environment within playbooks and can cause scripts to fail.

I see this error trying to execute a Python 3 script from within an ansible playbook, failing on a PyYAML module conflict:

Affected playbook tasks:

    - name: Run Python 3 script
      local_action: "shell python3 ../../utils/scripts/example_script.py"
      register: py3_test

Output:

TASK: [Get new host info] *****************************************************
failed: [localhost -> 127.0.0.1] => {"changed": true, "cmd": "python3 ../../utils/scripts/example_script.py", "delta": "0:00:00.043137", "end": "2015-01-07 17:04:02.318020", "rc": 1, "start": "2015-01-07 17:04:02.274883", "warnings": []}
stderr: Traceback (most recent call last):
  File "../../utils/scripts/example_script.py", line 9, in <module>
    from shared import dsc_shared
  File "/Users/bkeroack/code/infrastructure-automation/utils/scripts/shared/__init__.py", line 4, in <module>
    import yaml
  File "/usr/local/Cellar/ansible/1.8.2/libexec/vendor/lib/python2.7/site-packages/yaml/__init__.py", line 2, in <module>
    from error import *
ImportError: No module named 'error'

FATAL: all hosts have already failed -- aborting

The same script runs normally when run from a bash shell (ie, not within an Ansible playbook). The playbook runs normally (ie, the Python 3 script runs without error) on an Ubuntu box with the same Ansible version.

[tdsmith]

Yep, our packaging is doing this. This will take some creativity to resolve.

A workaround may be using python3 -E in your shebangs, which will keep python3 from respecting PYTHONPATH.

It's possible that instead of setting PYTHONPATH we could write shims similar to the one we use to invoke setup.py's that would modify sys.path and then execute the ansible scripts in the same interpreter, which solves the problem of our hackery being inflicted on child processes.

[MikeMcQuaid]

Yes, I agree that python3 -E is the way forward.

It's possible that instead of setting PYTHONPATH we could write shims similar to the one we use to invoke setup.py's that would modify sys.path and then execute the ansible scripts in the same interpreter, which solves the problem of our hackery being inflicted on child processes.

I'd rather avoid this approach as it's more complex and thus will be more error-prone and harder to maintain.

[bkeroackdsc]

Wait, if I'm understanding you correctly, you're proposing that users change all the python scripts called by their Ansible tasks to disregard PYTHONPATH? This is a non-starter. The same playbooks and scripts should work on OS X just like other platforms.

It's worth noting that my scripts don't even have a shebang--they are executed by calling python directly ("python3 ../path/to/script.py").

I'm not even clear on why the formula needs to vendorize modules, but then again I'm not at all familiar with Homebrew internals. I feel like it should depend upon other formulas that install the requisite modules in the Homebrew PYTHONPATH (/usr/local/Cellar/python3/...).

[MikeMcQuaid]

Well, users can do that or they use pip install ansible. We provide some Python applications in Homebrew that are also in pip but we need to package them in such a way that there are caveats.

[tdsmith]

It's worth noting that my scripts don't even have a shebang

Not trying to be difficult, but what makes this notable? Does this prevent you from invoking python with -E?

I'm not even clear on why the formula needs to vendorize modules, but then again I'm not at all familiar with Homebrew internals. I feel like it should depend upon other formulas that install the requisite modules in the Homebrew PYTHONPATH (/usr/local/Cellar/python3/...).

Your feeling is probably transferred from other packaging systems which work this way; Homebrew does not work this way, for reasons including avoiding the overhead of packaging libraries without many dependents that are already handled well by language-specific packaging tools.

[bkeroackdsc]

Not trying to be difficult, but what makes this notable? Does this prevent you from invoking python with -E?

Fair enough. I'm not trying to be difficult either (perhaps my tone was a bit aggressive in the previous response).

Your feeling is probably transferred from other packaging systems which work this way; Homebrew does not work this way, for reasons including avoiding the overhead of packaging libraries without many dependents that are already handled well by language-specific packaging tools.

Also a fair point.

If it isn't feasible to fix this issue in the Ansible formula directly, would it be a suitable compromise to add some big warnings to users after installation that playboook-executed Python scripts may break, and if that is a problem to install Ansible via pip instead?

[tdsmith]

Playbooks will execute in an environment where PYTHONPATH is set. If this causes problems, modifying your playbooks to invoke python with -E will cause your scripts to ignore PYTHONPATH.?

[iheitlager]

See my issue #35914. It might have to do with the rewrite of the shebang from the original #!/usr/bin/env python (preferred) to the #!/usr/bin/python (not preferred)

[edrozenberg]

I reference this issue in my Ansible issue regarding the Ansible pip module getting "confused" between the Ansible Homebrew site-packages and the main /usr/local site-packages. It appears this topic explains the problem I was having. In the end I uninstalled the ansible homebrew and did:

pip2 install --upgrade ansible

[tdsmith]

Yes, that's because pip thinks it's okay to remove any existing packages it finds in sys.path when it's installing a new version of the package. We can't readily work around it. Sorry!

[jacknagel]

Are there action items here, or is this wontfix (or something else)?

[johnscancella]

Are there any plans to fix this? This is causing my playbook to fail cause Ansible can't find the required modules it needs, specifically MySQLdb which is needed for the mysql_db (http://docs.ansible.com/mysql_db_module.html#notes)

[MikeMcQuaid]

No, unfortunately not. For your particular case @johnscancella submit a PR to include that module to Ansible.