Openstack-Ansible integration

[jrosser]

Not so much a bug but a placeholder for designing how OSA + Mitogen should work together.

TODO

Container connection
In order to get an AIO to build correctly, this needs Mitogen to be able to take the place of the OSA custom connection plugin, for a deployment which runs against localhost, with LXC containers on localhost. The OSA inventory contains enough info to find the name of the container to use for lxc-attach/nsenter, but currently Mitogen picks a different field from the ansible host data to connect to and fails.
Done - this is resolved by the issue510 branch (01840fd)

Handling of mitogen_via
Second, get a multinode deployment working, by providing a means for Mitogen to understand the physical_host OSA variable to do something like drive mitogen_via with the value. Would be great to not break the ability to have ssh bastions/jump hosts in the way and maintain the benefits of Mitogen in those scenarios.
Done - OSA inventory manipulation is the proper place to do this, and we have a proof-of-concept that works on the OSA side

delegate_to errors
See #496

Templating a change to the python interpreter
https://pasted.tech/pastes/b768a5dbe55352a89bd9a402b15f53f3218cfc5c

[dw]

Hi Jonathan :) #496 is important for OSA too -- it uses delegate_to heavily.

I'm hoping to get 0.2.4 out this weekend (famous last words), and will be able to look at new stuff straight after. Sorry for the huge delay on this -- one pair of hands :)

I am hoping to land 496 in 0.2.5, it's a huge pain for many-target, and it's quite a straightforward change that also allows me to refactor some insanely messy related code.

[cloudnull]

WIth https://review.openstack.org/#/c/591236 - I actually have it all working within OSA (also famous last words), however, if there was a key we could use instead of ansible_host to lookup container names when connecting to them it would be amazing. The simple inventory changes I've made to that PR made it possible to use mitogen_via within meta group vars however we set ansible_host to the ip address of the target machine which could be both a physical host or a container. If we had something like a mitogen_container_name be the default, which falls back to ansible_host, it would be amazing.

Within that patch I've resetting the ansible_host value to the inevntory_hostname but this is rather ugly and would have other ramifications throughout our roles (https://pasted.tech/pastes/aaed348e40074e61cf73e6e525528e828faef7ff).

[cloudnull]

Logs from my update in my multi-node lab environment - https://pasted.tech/pastes/c99b39913c59f3c04f92c7f385fbd0bef0a57ab6

so it seems to be working with the change to our inventory, however, as mentioned, having a mitogen specific container name key would make things a lot easier.

[dw]

Hi Kevin! mitogen_container_name it is :)

I've just pushed an issue510 branch. Please let me know if this isn't what you want.

[cloudnull]

Checking the branch out now.

[cloudnull]

on the recent run I'm seeing - https://pasted.tech/pastes/d8c239d5999a410b7301b33211759dc0b835373b.raw

• It would seem like there might be an issue with tasks that use delegation: example task - https://github.com/openstack/openstack-ansible-os_keystone/blob/master/tasks/keystone_service_update.yml#L26-L30

• Additionally, when testing our master branch I found there might be an issue with templateing a change to the python interpreter: https://pasted.tech/pastes/b768a5dbe55352a89bd9a402b15f53f3218cfc5c

I just recently changed my playbooks to the HEAD of osa master so there are some changes there that I didn't have on the previous run.

---

Edit: While this is an issue in our setup (OSA), it doesn't pertain to this issue.

[cloudnull]

Hi Kevin! mitogen_container_name it is :)

I've just pushed an issue510 branch. Please let me know if this isn't what you want.

This seems like a better implementation of the PR I had ( #512 ) 👍 from me.

[cloudnull]

I've opened - #515 - which addresses the python interpreter issues we're seeing

[cloudnull]

So PR #515 resolves the issue I was seeing with the python interpreter changing at the task level and allows the python interpreter to be a variable.

The issue I'm now is that there seems to be some JSON decode that's happening within the runner / backend? I'm really not sure about it though; still getting familiar with the code base.

Full debug output: [ https://pasted.tech/pastes/986f04377ac27504b39ef4b31178aefaf7709f4b.raw ]

Focused stacktrace: [ https://pasted.tech/pastes/4d3813238caef189b28e71faf28dacc16e0167f3 ]

[cloudnull]

Another thing that cropped up recently in testing, hundreds of runs, I found left over processes on the system.

Example

#   ps auxf | grep mitogen
root     20491  0.0  0.0  14428  1040 pts/0    S+   05:27   0:00          \_ grep --color=auto mitogen
root     15048  0.5  0.1 1385780 24976 ?       Ssl  05:25   0:00 /opt/ansible-runtime/bin/python2(mitogen:root@utility:14859)
root     15073  1.9  0.2 1432628 38192 ?       Sl   05:25   0:02  |   \_ /usr/bin/python(mitogen:root@utility:15048)
root     16008  1.7  0.3 1455496 50036 ?       Sl   05:25   0:01  |   \_ /usr/bin/python(mitogen:root@utility:15048)
root     17432  0.3  0.1 1383512 19900 ?       Sl   05:26   0:00  |   \_ /usr/bin/python(mitogen:root@utility:15048)
root     19769  0.4  0.1 1399892 22252 ?       Sl   05:26   0:00  |   \_ /usr/bin/python(mitogen:root@utility:15048)
root     19915  2.9  0.3 2981068 54480 ?       Sl   05:26   0:00      \_ /openstack/venvs/utility-19.0.0.0b1/bin/python(mitogen:root@utility:15048)
root     15299  0.1  0.1 1384416 19560 ?       Ssl  05:25   0:00 /opt/ansible-runtime/bin/python2(mitogen:root@utility:14859)

The processes were easy enough to shut down however they don't seem to be reliably cleaned up.