Skip to content

Fix ansible module dependencies#443

Merged
arianvp merged 2 commits intodevelopfrom
fix-python-path-for-good
Apr 6, 2021
Merged

Fix ansible module dependencies#443
arianvp merged 2 commits intodevelopfrom
fix-python-path-for-good

Conversation

@arianvp
Copy link
Copy Markdown
Contributor

@arianvp arianvp commented Apr 1, 2021

What we were doing before was way too complicated. Ansible itself
doesn't have any dependency on boto to function. The target host
needs to have boto installed.

In our case the target host is 'localhost'. Ansible will by default
pick /usr/bin/python3 as the python interpreter ogit n target hosts but of
course we can't rely on that having boto installed.

Instead, we configure ansible_python_interpreter to point to the
environment we built with nix; which contains a python that has boto
installed.

This way boto can be found and the playbooks should succeed!

@arianvp arianvp force-pushed the fix-python-path-for-good branch from f5d401f to e9806ef Compare April 1, 2021 09:50
Copy link
Copy Markdown
Contributor

@lucendio lucendio left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but DH action for macos does not agree.

@arianvp
Copy link
Copy Markdown
Contributor Author

arianvp commented Apr 1, 2021

This fixes it locally but it's still broken in the dockerfile. I'll try to get that to work Tuesday first.

@arianvp arianvp force-pushed the fix-python-path-for-good branch 3 times, most recently from dd0a560 to 8ec305b Compare April 6, 2021 08:35
@arianvp arianvp requested a review from lucendio April 6, 2021 08:35
@arianvp arianvp force-pushed the fix-python-path-for-good branch 2 times, most recently from 17d14ea to 7f3be3b Compare April 6, 2021 08:44
@arianvp
Copy link
Copy Markdown
Contributor Author

arianvp commented Apr 6, 2021

I tested this with the following playbook:

- hosts: localhost
  gather_facts: yes
  serial: 1
  become: no
  tasks:
    - debug: var=ansible_python_interpreter
    - debug: var=ansible_python_version
    - debug: var=ansible_playbook_python
    - aws_s3:
        mode: list
        bucket: public.wire.com
PLAY [localhost] *************************************************************************************************************

TASK [Gathering Facts] *******************************************************************************************************
ok: [localhost]

TASK [debug] *****************************************************************************************************************
ok: [localhost] => {
    "ansible_python_interpreter": "$LOCALHOST_PYTHON"
}

TASK [debug] *****************************************************************************************************************
ok: [localhost] => {
    "ansible_python_version": "3.8.8"
}

TASK [debug] *****************************************************************************************************************
ok: [localhost] => {
    "ansible_playbook_python": "/nix/store/yl69v76azrz4daiqksrhb8nnmdiqdjg9-python3-3.8.8/bin/python3.8"
}

TASK [aws_s3] ****************************************************************************************************************
ok: [localhost]

PLAY RECAP *******************************************************************************************************************
localhost                  : ok=5    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   


What we were doing before was way too complicated. Ansible itself
doesn't have any dependency on boto to function. The _target host_
needs to have boto installed.

By default the implicit localhost sets ansible_python_interpreter to
ansible_playbook_python.  This used to work but stopped working. (Or
maybe it never worked?) The python that ansible runs the playbooks as
cant find boto.  I don't know what changed. Maybe how nix assembles
python packages has changed.

Instead, we configure ansible_python_interpreter to point to the
environment we built with nix explicitly; which contains a python that
has boto installed.

This way boto can be found and the playbooks should succeed!

We could use the same simplification in hegemony I think.
@arianvp arianvp force-pushed the fix-python-path-for-good branch from 7f3be3b to 8612f08 Compare April 6, 2021 09:01
This resolves the complete path.  Makes debugging a bit easier. And
without this it didn't work on Gregor's machine (Reason still unclear to
me)
@arianvp arianvp requested a review from lucendio April 6, 2021 10:28
@arianvp arianvp merged commit 5b05ffe into develop Apr 6, 2021
@arianvp arianvp deleted the fix-python-path-for-good branch April 6, 2021 10:35
arianvp added a commit that referenced this pull request Apr 16, 2021
* Fix ansible module dependencies

What we were doing before was way too complicated. Ansible itself
doesn't have any dependency on boto to function. The _target host_
needs to have boto installed.

By default the implicit localhost sets ansible_python_interpreter to
ansible_playbook_python.  This used to work but stopped working. (Or
maybe it never worked?) The python that ansible runs the playbooks as
cant find boto.  I don't know what changed. Maybe how nix assembles
python packages has changed.

Instead, we configure ansible_python_interpreter to point to the
environment we built with nix explicitly; which contains a python that
has boto installed.

This way boto can be found and the playbooks should succeed!

We could use the same simplification in hegemony I think.

* Use lookup('env')

This resolves the complete path.  Makes debugging a bit easier. And
without this it didn't work on Gregor's machine (Reason still unclear to
me)
@@ -1,11 +1,13 @@
env="$(nix-build $PWD/default.nix -A env --no-out-link)"
nix-build $PWD/default.nix -A env --out-link .nix-env
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

drive-by comment (feel free to ignore) this is a nice .envrc and Nix pattern :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants