Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ansible runner issues #431

Closed
philpep opened this issue Apr 29, 2019 · 1 comment
Assignees

Comments

@philpep
Copy link
Owner

@philpep philpep commented Apr 29, 2019

Here I sumarize issues related to the ansible backend runner.

Ansible has been available as a "connection backend", meaning the use of ansible python api to execute commands on remote hosts. This have several issues:

  • The code is very hard to maintain because ansible doesn't have a stable python API. From https://docs.ansible.com/ansible/latest/dev_guide/developing_api.html "This API is intended for internal Ansible use. Ansible may make changes to this API at any time that could break backward compatibility with older versions of the API. Because of this, external use is not supported by Ansible." We are currently no longer compatible with ansible 2.8

  • We are actually violating the license because ansible is GPL and testinfra is not (apache2), so importing ansible in python require the code to be under GPL.

Requirements:

  • I want to exclude all solutions including writting code with the ansible python API. Every new release of ansible produce a large amount of work and frustration to me.

  • Fully backward compatible, at least for general usage (molecule should not break)

  • Should be fast (at least no slower than now).

Possible solutions:

https://github.com/ansible/ansible-runner first appeared to be a possible solution, but after reading the documentation and a bit of code, I think it's actually not providing the feature we need (note that I might have missed something...)

I considered interfacing with ansible cli directly, but this will introduce a terrible performance regression.

I think people using testinfra and ansible use the ansible backend for "convenience", because it can read the inventory, but only some tests actually require ansible (only those using the ansible module). Also I think most people use ansible with either local, ssh or docker connection and testinfra already have such connection backends.

So I propose to:

  • Write some code to parse/decode ansible configuration and inventory and convert them to testinfra local, ssh or docker backend. ansible-config and ansible-inventory tools can help here.

  • Reimplement the ansible module using ansible cli called in subprocess.

This will drop support for other ansible connection plugins (https://docs.ansible.com/ansible/latest/plugins/connection.html), but I think we can implement them later, we already have some code for kubectl, lxc, winrm.

I already have an POC implementation of this and it works pretty well. I'll submit a PR soon.

Please leave a comment if you want to point me on your use case, I'll be happy to test implementation with your code !

Also special thanks to people that have tried to fix this issue by submitting PRs, especially @jctanner and @ianw , your work helped me a lot for sumarizing this issue and find this solution !

Related to #428 #410 and #401

@philpep philpep self-assigned this Apr 29, 2019
@jctanner

This comment has been minimized.

Copy link

@jctanner jctanner commented Apr 29, 2019

@philpep fair enough, but i'm still curious if you could elaborate further on what is missing from runner.

philpep added a commit that referenced this issue Apr 29, 2019
Following specification on #431

* Use ansible-inventory cli to parse inventory, group and host variables

* Ansible backend re-use existing backend (local, ssh) to run commands.
  This is a breaking change because we do not support all connection
  backends from ansible.

* The Ansible module run with the ansible cli.

* Add more tests

* Fix the skipped "encoding" test which pass now.

This fix maintainability issue with ansible and the license issue
(ansible is GPL).

Closes #431
philpep added a commit that referenced this issue Apr 30, 2019
Following specification on #431

* Use ansible-inventory cli to parse inventory, group and host variables

* Ansible backend re-use existing backend (local, ssh) to run commands.
  This is a breaking change because we do not support all connection
  backends from ansible.

* The Ansible module run with the ansible cli.

* Add more tests

* Fix the skipped "encoding" test which pass now.

This fix maintainability issue with ansible and the license issue
(ansible is GPL).

Closes #431
philpep added a commit that referenced this issue Apr 30, 2019
Following specification on #431

* Use ansible-inventory cli to parse inventory, group and host variables

* Ansible backend re-use existing backend (local, ssh) to run commands.
  This is a breaking change because we do not support all connection
  backends from ansible.

* The Ansible module run with the ansible cli.

* Add more tests

* Fix the skipped "encoding" test which pass now.

This fix maintainability issue with ansible and the license issue
(ansible is GPL).

Closes #431
philpep added a commit that referenced this issue May 4, 2019
Following specification on #431

* Use ansible-inventory cli to parse inventory, group and host variables

* Ansible backend re-use existing backend (local, ssh) to run commands.
  This is a breaking change because we do not support all connection
  backends from ansible.

* The Ansible module run with the ansible cli.

* Add more tests

* Fix the skipped "encoding" test which pass now.

This fix maintainability issue with ansible and the license issue
(ansible is GPL).

Closes #431
@philpep philpep closed this in 0c386d8 May 5, 2019
kentr added a commit to kentr/drupal-vm that referenced this issue May 19, 2019
`testinfra` >= `3.0` is required for Ansible `2.8`.
See philpep/testinfra#431
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.