documentation: passing dynamic role variables to tests when runnning molecule verify

[decentral1se]

I know this could be batted away as something that is related to testinfra only, however, I am finding that I would like to ask here - would you accept a documentation PR that outlines a strategy for including role variables that you passed to your converge playbook, into your unit tests?

Here's what I am currently doing:

# molecule/default/playbook.yml
---
- name: Converge
  hosts: all
  vars_files:
    - testvars.yml
  roles:
    - role: my-role
      foobar: "{{ test_foobar }}"

Then in a molecule/default/testvars.yml, I have:

---
test_foobar: barfoo

Then, in my molecule/default/tests/conftest.py, I do the following:

import os

import pytest
from testinfra.utils.ansible_runner import AnsibleRunner

DEFAULT_HOST = 'all'

inventory = os.environ['MOLECULE_INVENTORY_FILE']
runner = AnsibleRunner(inventory)
runner.get_hosts(DEFAULT_HOST)


@pytest.fixture
def testvars(host):
    variables = runner.run(
        DEFAULT_HOST,
        'include_vars',
        'testvars.yml'
    )
    return variables['ansible_facts']

And in my tests, I can then:

def test_something(host, testvars):
    print(testvars['test_foobar'])

As far as I can see, this is the cleanest way (I've digged around in a lot of tickets) of matching up the dynamic variables that you pass to your role and your testinfra pytest tests.

If it isn't, please someone tell me :)

In any case, the main question remains - is there a place we can start to document this on the RTD setup?

[decentral1se]

Woops, meant to open this on https://github.com/metacloud/molecule, closing! :)

[decentral1se]

Following ansible/molecule#1396 (comment), I'm not getting anywhere on the molecule repository. Could you provide any guidance here or could we suggest some sort of an API for this? I'd be happy to implement something.

A simple use case to focus the discussion:

I pass in a variable to my role called http_port which I open on the firewall with ufw
I test that the vaule of http_port passed into the role was opened by ufw
How do I access this http_port value in my test infra tests?

[philpep]

Testinfra should load host and group variables automatically. What do you mean by "I pass a variable to my role ?", is this a role "default" variable ?
I think you can load such variables by calling the proper api from ansible (see https://github.com/philpep/testinfra/blob/master/testinfra/utils/ansible_runner.py)

[decentral1se]

What do you mean by "I pass a variable to my role ?", is this a role "default" variable ?

When I do this:

# molecule/default/playbook.yml
---
- name: Converge
  hosts: all
  vars_files:
    - testvars.yml
  roles:
    - role: my-role
      foobar: "{{ test_foobar }}"

So, here, I meant that I pass a value to foobar. I want to be able to access foobar in my test.

I think you can load such variables by calling the proper api from ansible

I haven't been able to see any value for foobar when I use runner.get_variables(), should I?

I've seen you mention that testinfra can't know about my playbooks but I assumed that it could pick up variables that are passed into the role. Hope that makes sense.

[barnabasJ]

I had a similar problem, because I wanted to access the role defaults and vars. I have the following in my conftest.py. I did it this way to keep the variable precedence of ansible intact. I used the cache because I had the problem that the host fixture was teared down and setup for every test which made this really slow.

import pytest

cache = {}

@pytest.fixture(scope='module')
def ansible_vars(request, host):
    role_name =  request.cls.__name__.replace("Test", "").lower()
    if (id(host), role_name) in cache.keys():
        return cache[(id(host), role_name)]

    ansible_vars = host.ansible("include_vars", "file=roles/%s/defaults/main.yml name=role_defaults" %
                                (role_name))["ansible_facts"]["role_defaults"]
    ansible_vars.update(host.ansible.get_variables())
    ansible_vars.update(host.ansible(
        "include_vars", "file=roles/%s/vars/main.yml name=role_vars" % (role_name))["ansible_facts"]["role_vars"])
    del ansible_vars['role_defaults']

    cache[(id(host), role_name)] = ansible_vars

    return ansible_vars

Seems like we had pretty much the same idea :)

[decentral1se]

Thanks for weighing in @barnabasJ! I am sure (from counting tickets on molecule and testinfra repositories) that this is a use case that many people need solved. I must say, your solution does look quite nice!

Perhaps this could be solved as a family of test fixtures that come packaged up.

Any thoughts @philpep?

[dangoncalves]

@barnabasJ : how do you use testinfra? It seems you don't use molecule. In my case, putting a conftest.py file in my test dir doesn't affect my tests. Moreover, I don't understand how you use your functions ansible_vars, more specifically I don't understand what is the request parameter. Could you be more explicit please?

[barnabasJ]

In my current environment it's hard to use docker or vms on my local machine, therefore I don't use molecule. But I write the tests to check if the servers are in the desired state after running ansible-playbooks. I just do it without molecule as a dev tool.
Testinfra is developed as a pytest plugin which allows us to use all the pytest features while writing our tests. When you write the tests you just specify host as an argument and it gets injected during runtime. This is because host is a pytest.fixture.
The conftest.py file is used to create fixtures that can be shared between different test files. While creating fixtures it's possible to inject other fixtures, which you can then use to get the functionality you want. request is another fixture built into pytest that allows to inspect the context from which the fixture is called.

[dangoncalves]

OK thanks for the explaination.
I was able to adapt your code to an usage with molecule. I just added the following code to my tests (it works like yours):

import pytest

@pytest.fixture
def get_vars(host):
    defaults_files = "file=../../defaults/main.yml name=role_defaults"
    vars_files = "file=../../vars/main.yml name=role_vars"

    ansible_vars = host.ansible(
        "include_vars",
        defaults_files)["ansible_facts"]["role_defaults"]

    ansible_vars.update(host.ansible(
        "include_vars",
        vars_files)["ansible_facts"]["role_vars"])

    return ansible_vars

Of course it could be very useful if this was integrated natively in Testinfra, but it could be not so easy because of the many use cases that users could encounter.

[hlarsen]

i'd definitely love to see something like this officially included, having only used molecule and testinfra for about a week it was one of my biggest gripes.

@dangoncalves that fixtures works great, thanks!

[barnabasJ]

There probably isn't one solution for everything. But if we had a documented example, I think most would be able to adapt it to their use-case.

[decentral1se]

Question: are there any downsides to this approach of using include_vars? If you see links above, the molecule core devs seem to think it can lead to false positives. I haven't seen any limitations in my work so far.

Otherwise, can we come to some agreement on adding variable fixtures to the https://github.com/philpep/testinfra/blob/master/testinfra/plugin.py? It appears since these paths are totally configurable, we'll need defaults (like ../../defaults/main.yml) and then allow to override (using some tricks from https://docs.pytest.org/en/latest/example/parametrize.html).

[codylane]

I'm still not sure I fully grasp the notion of why you need to pass variables around like that and why you need them available for your integration tests? I've read this issue multiple times and I cannot come up with a good reason to do this without it being overly complex and confusing for the person maintaining such a library. I think I would need to see an example repository with these additions included to see why it might be worth while effort to make a recommendation.

I know the Ansible community does not believe unit testing but I feel that this scenario, testing variable combinations in custom roles to be more a unit test not an integration test. Testinfra is a integration test library and I could see adding such a new fixture would cause the test library to behave "weirdly" or "falsey" at times if the user has mis-configured their environment or has files laying around.

If I understand your problem scenario then one of the ways I've solved something like this is to just create a new molecule scenario and add those custom variables in the converge.yml playbook. Yes, this will increase the time it takes to test your roles, but it's a small price for not testing or having have to many test fixtures floating around and causing you more grief down the road. Here's the example repo

Anyway, I really like this discussion and curious to learn more about how other folks are doing things.

[decentral1se]

I'm still not sure I fully grasp the notion of why you need to pass variables around like that

I have laid out a clear example in #345 (comment):

A simple use case to focus the discussion:

I pass in a variable to my role called http_port which I open on the firewall with ufw
I test that the vaule of http_port passed into the role was opened by ufw
How do I access this http_port value in my test infra tests?

I could see adding such a new fixture would cause the test library to behave "weirdly" or "falsey" at times if the user has mis-configured their environment or has files laying around.

You're right, we need to avoid this. The default should always work.

If I understand your problem scenario then one of the ways I've solved something like this is to just create a new molecule scenario and add those custom variables in the converge.yml playbook. Yes, this will increase the time it takes to test your roles, but it's a small price for not testing or having have to many test fixtures floating around and causing you more grief down the road. Here's the example repo

Looking at https://github.com/codylane/ansible-role-pyenv/blob/master/molecule/debian/playbook.yml, you have specified - 2.6.9, - 2.7.15 - 3.6.6 as your pyenv versions you want to install. In your https://github.com/codylane/ansible-role-pyenv/blob/517cd3ab912a2c2cf0c62da485e7976cf55555b2/molecule/debian/tests/test_default.py#L114, you hard code which versions you expect to be installed. In fact, (for whatever reason), you're not testing that the version 3.6.6 directory is present.

This example is proving (IMHO) why we need to be able to access role variables in the tests.

If you could access the pyenv versions values you passed to the role in the test, then you could simply iterate over them and make sure the directories are present. This would mean that your tests stay synced with your role variables.

[barnabasJ]

In my case we would also like to be able to run the test periodically on the servers after provisioning them. But not all the servers are exactly the same. This means hard coding the tests is not always possible. If I use the default_vars during development of the ansible roles. I can just specify the vars in the group_vars/host_vars later the way I would do it if I ran a playbook with multiple roles on multiple servers and the defaults are overwritten and the tests are specific to the servers they're run on.

[codylane]

Ahh, I see both your points now. Thank you for taking the time to help clarify.

lwm - I will fix that missing test, thank you for pointing it out. It was there, then I did a refactor due to travis "timeout" problems and I must have forgot to add it back in.

I've really enjoyed this discussion. I'd like to see if I can take a stab at what I think is being proposed here.

Requirement

When using testinfra to test our ansible roles, we would like to have a way to pass in a group of variables that work for the following cases:

1. These variables are passed to our role when we run pytest which configures our play.
2. These variables are available to our system under test in our test_*.py where they can also be used to avoid hard coding them in our tests?

I could also see a need where the scope needs to also be a requirement to avoid the "setup" and "teardown" routines. Following the testinfra paradigm there are "module" and "function" scope test fixtures so we would probably want a way to do both for maximum flexibility.

Question

1. I've not tried this but can we not use group_vars or host_vars via the ansible_runner?

Potential Solution Discussion

I wonder if we could explore the following scenarios?

1.) See if we can get ansible_runner to use a dynamic inventory where the dynamic inventory is a JSON blob of configuration. (I'll provide an example of this when I have more time)

2.) I also wonder if it is possible to use group_vars or host_vars which should be available to both ansible and testinfra?

3.) I wonder if we create a pytest fixture to push custom facts to the SUT (system under test) which would be available to both ansible and testinfra? I can also provide an example of this if anyone is interested.

[barnabasJ]

Just a quick comment on group_vars/host_vars. Yes the ansible_runner picks up on them but when creating roles I don't use those, I see them as a higher level tool used with playbooks.
I feel that roles should be runable/testable with just the defaults in most cases and I use group/host_var to customize the playbook runs for the different target servers.

What do you think?

[decentral1se]

Thanks! I wonder, as another idea, if we can't get this under https://testinfra.readthedocs.io/en/latest/modules.html#testinfra.modules.ansible.Ansible.get_variables since that is the existing API. That leads into https://github.com/philpep/testinfra/blob/master/testinfra/utils/ansible_runner.py which does use the official Ansible module - perhaps there is something we can do there. I will take a look when I get time.

[codylane]

barnabasJ - I hear you and that makes a lot of sense from a testing perspective. My first thought then is to use a dynamic inventory script (i'm still working on an example) that is configured via a JSOB blob where each group would be a different test scenario of configuration data which is then passed to ansible.

However, the challenge is trying to get at those variables through the 'ansible' library API and it's been a few years since I got intimate with it. It will take a few days as I have free time to sort it out but I've got a few more ideas to try.

lwm - Yup, in my initial testing get_variables is close but it's missing info that I know we can get at. I just need to massage it a bit to get what we want because hostvars will give us pretty much everything you both are after.

[codylane]

From initial testing, I think I understand why we cannot see the role variables it's because the ansible_runner does not see a playbook, instead it wraps what a playbook might look like via dictionary object

So, with that said, we will need to introduce some new features for the ansible_runner to read to the following:

1. It should support reading a playbook YAML file from the a directory inside the project root.
2. There may be multiple playbook files that need to be read for scenario testing where each environment is different and or otherwise isolated.
3. The variables contained within that playbook should be returned via the get_variables() routine of the ansible_runner so that the system under test (SUT) will be able to interrogate and or simplify our tests from having hard coded tests.

The result that should be returned is a new dictionary that represents hostvars + facts + inventory vars, defaults, or any other variable passed to a playbook run and it should go out of scope depending on the function scope being tested i.e. (module, session, function).

However, before we introduce this new feature, I'd like to get consensus from the project owner @philpep if this solution would be acceptable to implement?

@lwm and @barnabasJ - Please also chime in if I've missed anything.

[decentral1se]

Great.

There may be multiple playbook files that need to be read for scenario testing where each environment is different and or otherwise isolated.

Should we use some naming convention to allow picking up and reading of vars files in the scenario folder?

However, before we introduce this new feature, I'd like to get consensus from the project owner

Good idea.

[barnabasJ]

The runner is attached to the host which is module scoped. Which should be good for most situations, I'm not sure about how this would effect roles developed with molecule, but the variables for a host should probably not change during a test run.

The playbook dictionary object you linked to, is located in the run function which is executed everytime something is run on the target. Parsing the playbook and variables every time a command is invoked is very expensive time-wise.
If you really need to reread the variable/playbook files this should have to be called explicitly, because of the performance issues. I would prefer to parse the necessary files during e.g. backend creation, so it only happens once per host.

I'm also not sure if we even need such a complicated setup. Are you guys trying to read variables from the playbook itself too?

Right now it's possible to specify the host from within a module using the
testinfra_host variable like:

testinfra_hosts = ["ansible://all?ansible_inventory=hosts.ini"]

Maybe it'would be enough to add another option here, like specifying a role dir with the defaults and vars to parse. But like you said it's probably best to wait and see what @philpep thinks.

[barnabasJ]

I did play around with it a little bit. Here is a minimal example. All the needed stuff is already in the ansible_runner except the playbook.

import ansible.cli.playbook
import ansible.playbook

import pprint

pp = pprint.PrettyPrinter()

cli = ansible.cli.playbook.PlaybookCLI(None)
cli.options = cli.base_parser(
    connect_opts=True,
    meta_opts=True,
    runas_opts=True,
    subset_opts=True,
    check_opts=True,
    inventory_opts=True,
    runtask_opts=True,
    vault_opts=True,
    fork_opts=True,
    module_opts=True,
).parse_args([])[0]
cli.normalize_become_options()
cli.options.connection = "smart"
cli.options.inventory = 'hosts.ini'
loader, inventory, variable_manager = (
    cli._play_prereqs(cli.options))
playbook = ansible.playbook.Playbook.load('playbook.yml', variable_manager, loader)
pp.pprint(variable_manager.get_vars(playbook.get_plays()[0]))

I just copied most of this from the AnsibleRunnerV2(AnsibleRunnerBase):.__init__(self, host_list): method. I just added the last 2 lines.

This is just thrown together and thought of as a proof of concept.

[codylane]

@barnabasJ - Thanks for posting that little snippet that is exactly what I was thinking, however, I wonder in your example if are also experiencing only seeing the role default vars?

For example, given playbook.yml

---

- hosts: all
  become: true

  roles:
    - role: codylane.pyenv
      pyenv_install_these_pythons:
        - 2.7.15
      pyenv_user: vagrant
      pyenv_group: vagrant
      pyenv_root: /home/vagrant/.pyenv

ansible.cfg

[ssh_connection]
pipelining = True


[defaults]
# vault_password_file = .vaultpw

inventory= ./inventory
error_on_missing_handler = True
ansible_managed = Ansible managed: {file} modified on %Y-%m-%d %H:%M:%S by {uid} on {host}
deprecation_warnings = True
display_skipped_hosts = True
host_key_checking = False

gathering = smart
# gather_subset = all
fact_caching = jsonfile
fact_caching_connection = tmp/ansible-facts
fact_caching_timeout = 1500

roles_path = roles

[diff]
always = yes
context = 10

requirements.yml

---

- src: https://github.com/codylane/ansible-role-pyenv
  version: master
  name: codylane.pyenv

ansible-galaxy install -r requirements.yml

cat foo.py

#!/usr/bin/env python
# flake8: noqa

import ansible.cli.playbook
import ansible.playbook

import pprint

pp = pprint.PrettyPrinter()

cli = ansible.cli.playbook.PlaybookCLI(None)
cli.options = cli.base_parser(
        connect_opts=True,
        meta_opts=True,
        runas_opts=True,
        subset_opts=True,
        check_opts=True,
        inventory_opts=True,
        runtask_opts=True,
        vault_opts=True,
        fork_opts=True,
        module_opts=True,
).parse_args([])[0]
cli.normalize_become_options()
cli.options.connection = "smart"
cli.options.inventory = 'inventory'
loader, inventory, variable_manager = (
        cli._play_prereqs(cli.options))
playbook = ansible.playbook.Playbook.load('playbook.yml', variable_manager, loader)
pp.pprint(variable_manager.get_vars(playbook.get_plays()[0]))

when I run python ./foo.py

{'ansible_check_mode': False,
 'ansible_diff_mode': True,
 'ansible_forks': 5,
 'ansible_inventory_sources': 'inventory',
 'ansible_play_batch': [u'foo-centos7', u'foo-ubuntu1404'],
 'ansible_play_hosts': [u'foo-centos7', u'foo-ubuntu1404'],
 'ansible_play_hosts_all': [u'foo-centos7', u'foo-ubuntu1404'],
 'ansible_playbook_python': '/Users/codylane/.pyenv/versions/testinfra-issue-345-2.7.15/bin/python',
 'ansible_run_tags': [],
 'ansible_skip_tags': [],
 'ansible_version': {'full': '2.5.5',
                     'major': 2,
                     'minor': 5,
                     'revision': 5,
                     'string': '2.5.5'},
 'groups': {'all': [u'foo-centos7', u'foo-ubuntu1404'],
            u'suts': [u'foo-centos7', u'foo-ubuntu1404'],
            'ungrouped': []},
 'omit': '__omit_place_holder__b70745f194898f5bfbe5e4900d6bb515963e148c',
 'play_hosts': [u'foo-centos7', u'foo-ubuntu1404'],
 'playbook_dir': u'/Users/codylane/prj/python/testinfra-issue-gists/issue-345',
 u'pyenv_activated_plugins': [{u'name': u'pyenv-virtualenv',
                               u'remote': u'origin',
                               u'repo': u'https://github.com/pyenv/pyenv-virtualenv',
                               u'update': True,
                               u'version': u'master'}],
 u'pyenv_git': {u'remote': u'origin',
                u'repo': u'https://github.com/pyenv/pyenv.git',
                u'update': True,
                u'version': u'master'},
 u'pyenv_group': u'root',
 u'pyenv_install_these_pythons': [u'2.6.9',
                                  u'2.7.15',
                                  u'3.4.8',
                                  u'3.5.5',
                                  u'3.6.6',
                                  u'3.7.0'],
 u'pyenv_profiled_script': u'/etc/profile.d/pyenv.sh',
 u'pyenv_root': u'/opt/pyenv',
 u'pyenv_user': u'root',
 'role_names': [u'codylane.pyenv'],
 'vars': {'ansible_check_mode': False,
          'ansible_diff_mode': True,
          'ansible_forks': 5,
          'ansible_inventory_sources': 'inventory',
          'ansible_play_batch': [u'foo-centos7', u'foo-ubuntu1404'],
          'ansible_play_hosts': [u'foo-centos7', u'foo-ubuntu1404'],
          'ansible_play_hosts_all': [u'foo-centos7', u'foo-ubuntu1404'],
          'ansible_playbook_python': '/Users/codylane/.pyenv/versions/testinfra-issue-345-2.7.15/bin/python',
          'ansible_run_tags': [],
          'ansible_skip_tags': [],
          'ansible_version': {'full': '2.5.5',
                              'major': 2,
                              'minor': 5,
                              'revision': 5,
                              'string': '2.5.5'},
          'groups': {'all': [u'foo-centos7', u'foo-ubuntu1404'],
                     u'suts': [u'foo-centos7', u'foo-ubuntu1404'],
                     'ungrouped': []},
          'omit': '__omit_place_holder__b70745f194898f5bfbe5e4900d6bb515963e148c',
          'play_hosts': [u'foo-centos7', u'foo-ubuntu1404'],
          'playbook_dir': u'/Users/codylane/prj/python/testinfra-issue-gists/issue-345',
          u'pyenv_activated_plugins': [{u'name': u'pyenv-virtualenv',
                                        u'remote': u'origin',
                                        u'repo': u'https://github.com/pyenv/pyenv-virtualenv',
                                        u'update': True,
                                        u'version': u'master'}],
          u'pyenv_git': {u'remote': u'origin',
                         u'repo': u'https://github.com/pyenv/pyenv.git',
                         u'update': True,
                         u'version': u'master'},
          u'pyenv_group': u'root',
          u'pyenv_install_these_pythons': [u'2.6.9',
                                           u'2.7.15',
                                           u'3.4.8',
                                           u'3.5.5',
                                           u'3.6.6',
                                           u'3.7.0'],
          u'pyenv_profiled_script': u'/etc/profile.d/pyenv.sh',
          u'pyenv_root': u'/opt/pyenv',
          u'pyenv_user': u'root',
          'role_names': [u'codylane.pyenv']}}

Which the above isn't quiet right, it's only returning the default vars not the vars overriden in my playbook.yml. I'll tinker with this some more, does anyone else also get this behavior?

Here's the place where I will be attempting my examples. https://github.com/codylane/testinfra-issue-gists/tree/master/issue-345

[decentral1se]

B U M P 🦈 🦈 🦈

[philpep]

Hi, I'm digging in this issue but I'm a bit confused because I'm not familiar with molecule and ansible.
Can someone summarize (shortly) what the current state of this issue ?

IIUC, you run ansible through molecule which run a playbook with "vars_files", "roles" and role variables, and you expect to have a way in testinfra tests to get theses variables (= merge of host_vars, group_vars, vars_file, role default vars and role overriden vars) ?

testinfra doesn't have access to the playbook the machine was provisioned with, it only read inventory, ansible.cfg and host/groups vars.
I think it should be possible to add a playbook parameter to host.ansible.get_variables() which could read the playbook and merge the variables just like ansible do by calling the ansible playbook python API. Do you think it will solve your problem ?

[decentral1se]

Hi, I'm digging in this issue but I'm a bit confused because I'm not familiar with molecule and ansible.
Can someone summarize (shortly) what the current state of this issue ?

Thanks for taking some time on this! We kind of got lost at sea 😖

IIUC, you run ansible through molecule which run a playbook with "vars_files", "roles" and role variables, and you expect to have a way in testinfra tests to get theses variables (= merge of host_vars, group_vars, vars_file, role default vars and role overriden vars) ?

Yes, you got it!

I think it should be possible to add a playbook parameter to host.ansible.get_variables() which could read the playbook and merge the variables just like ansible do by calling the ansible playbook python API. Do you think it will solve your problem ?

I think, basically, we decided that it would be the nicest if we could extend the get_variables() interface, so this sounds like it would work! If I pass a bunch of vars into my playbook and I can pass my playbook into the function and get those vars, we're good.

[barnabasJ]

First of all, I'd like to thank you too for spending your time on this.

I don't use molecule because I can't use docker on my work machine at the moment. During tests, I also like to use ansible variables though. But to test the roles in isolation, it's only possible for me to use the variables inside the role (defaults and vars). Therefore, I don't have a playbook for every role.

I took a glance at the ansible code and saw that the smallest unit for getting the variables is a play. A playbook could potentially hold multiple Plays which might make it difficult to load the correct variables from every playbook. We would probably need to specify at least the playbook and optionally which play we want to load.

For my specific use case, it would also be nice if I could just inline the necessary play data as a string or a dictionary and don't have to create a playbook for every role. Specifically, because I also run the tests of multiple roles combined to test a specific target host which had multiple plays run against it and the legacy roles I work with use the same variable name from time to time. Which would be problematic because some roles would overwrite values for other roles.

[philpep]

Yes, I think role vars are scoped to the role.
So what do you think of this ? :

get_variables(playbook=<file or dict or yaml string>) --> would return merge of host_vars/group_vars and vars_file defined in playbook
get_variables(playbook=..., role=<string of a role name>) --> would return merge of host_vars/group_vars, vars_file AND role default vars + overriden role vars

It might be hard to write this, depending on the complexity of the ansible API, and by experience, it is so I think it's ok to not handle this for old ansible versions (and raise a NotImplementedError).

[decentral1se]

Right, apologies @barnabasJ, I see we have differing needs.

It might be hard to write this, depending on the complexity of the ansible API, and by experience, it is so I think it's ok to not handle this for old ansible versions (and raise a NotImplementedError).

<3

I've still not had time to do anything to make this happen but I promise I will QA test it :)

[barnabasJ]

@lwm no worries,

I like your idea @philpep, and I think the proposed solution should fit most needs very well. I also agree that it's not necessary to implement this for older ansbile versions.

Let me know if I can do anything to help.

[camillehuot]

Also looking forward this capability :)

[sebastienGuavus]

Not sure if a lot have changed in the code base since the discussion here have taken place but the variable_manager in the AnsibleRunner should take care of caching the variable. The only thing I would change in testinfra would be to add the possibility to resolve templated variable.

This is the fixture I put in my molecule test file. Hope it can be useful to someone.

from ansible.template import Templar
from ansible.parsing.dataloader import DataLoader

@pytest.fixture(scope='module')
def ansible_vars(host):
    """
    Return a dict of the variable defined in the role tested or the inventory.
    Ansible variable precedence is respected.
    """
    defaults_files = "file=../../defaults/main.yml"
    vars_files = "file=../../vars/main.yml"

    host.ansible("setup")
    host.ansible("include_vars", defaults_files)
    host.ansible("include_vars", vars_files)
    all_vars = host.ansible.get_variables()
    all_vars["ansible_play_host_all"] = testinfra_hosts
    templar = Templar(loader=DataLoader(), variables=all_vars)
    return templar.template(all_vars, fail_on_undefined=False)

[decentral1se]

Oh nice @sebastienGuavus! @philpep, do you have any indication on what kind of change you'd accept for this issue? I'd gladly put some time in on a pull request. Still would really like to see a more convenience API available for this in testinfra 👍

[RebelCodeBase]

I've created a pull request to pass kwargs to ansible in order to write a python module which resolves and exposes role and testinfra variables as a tester fixture. I've outlined my idea with code examples in the pull request.

[RebelCodeBase]

The testaid pytest plugin should solve this problem. Originally, it was calling the ansible cli interface using the testinfra host fixture. But after the latest rewrite it is using the ansible python api directly and even exposes it in two fixtures. As testaid is independent from the host fixture it can now use the session scope which allows pytest to cache the testvars.

[piroux]

One year later, unfortunately, as of almost mid-2020, the testaid project has still many limits and cannot be used reliably. Moreover it seems out of maintenance and the author seems to indicate that it was merged/moved to another repo.

I think the solution should be found in the testinfra repo directly.
What are the current blocking points to advance this issue here ?
How can we help ?

[stephan-devop]

Indeed testaid has been moved to takelage-var and is under active development.

With yesterday's relase 0.4.0, takelvars has become a special case of multitestvars which now gathers the variables for multiple hosts.

The transition from testaid to takeltest (which is now part of a larger framework but can be used independently) led to yet another major refactoring. Hopefully, a lot of issues have been resolved but feel free to post any ongoing issues again in the new repository.