Got Ansible own modules? Run them from within the Salt just like that!

[isbm]

What does this PR do?

In case you want to migrate away from Ansible, but have already modules written for it — no worries. Just toss them in and run them from Salt and Salt states.

Limitations

Currently, only Python-written modules are supported. Support for other languages are still in progress.

Requirements

The entire Ansible should be installed on the Minion. No remote execution at the moment. Continuation of this work can pull required files from the Ansible through the Salt transport. These files should be installed on Master only (in a future) and then thus get executed on the Minions.

Description and Usage

Normally Salt modules are usually module.function. That is, a typical call of a function from the module will be: salt hostname module.function some=parameters. But since we are connecting the entire Ansible world to the Salt universe, we would like to keep the namespace as this is inside the Ansible. That said, any Ansible module is placed in the sub-directory that potentially can clash with other same-named module. Regardless the future plans of Ansible community, we already have a namespace introduced. Therefore any Ansible module is called with a full "path", prefixed with ansible:

salt \* ansible.system.ping data="Hello from Ansible via Salt"

If you use typical development environment, this will result to the following:

saltdevel:
    ----------
    ping:
        Hello from Ansible via Salt

Listing available Ansible modules

In order to list all available Ansible modules, call ansible.list:

salt yourminion ansible.list

To filter them out, just type a string that matches inside them, e.g. "linux":

$ salt saltdevel ansible.list info
saltdevel:
    - net_tools.ipinfoio_facts
    - network.cloudengine.ce_info_center_debug
    - network.cloudengine.ce_info_center_global
    - network.cloudengine.ce_info_center_log
    - network.cloudengine.ce_info_center_trap

Getting more information about the module

You can read all the details about any Ansible modules by issuing ansible.help. For example:

$ salt saltdevel ansible.help net_tools.ipinfoio_facts
saltdevel:
    ----------
    Available sections on module "net_tools.ipinfoio_facts":
        - version_added
        - author
        - short_description
        - notes
        - options
        - module
    Description:
        - Gather IP geolocation facts of a host's IP address using ipinfo.io API

At this point you may ask for short_description, notes etc. For example, to know how to use it, call options:

$ salt saltdevel ansible.help net_tools.ipinfoio_facts options
saltdevel:
    ----------
    options:
        ----------
        http_agent:
            ----------
            default:
                ansible-ipinfoio-module/0.0.1
            description:
                - Set http user agent
            required:
                False
        timeout:
            ----------
            default:
                10
            description:
                - HTTP connection timeout in seconds
            required:
                False

Calling Ansible modules

Calling Ansible modules are just like any other Salt modules, straight-forward, except they will be needing to stay within their own namespace, which is different than we are using typically in Salt:

$ salt saltdevel ansible.net_tools.ipinfoio_facts                                    
saltdevel:
    ----------
    ansible_facts:
        ----------
        city:
            Nuremberg
        country:
            DE
        hostname:
            charybdis-ext.suse.de
        ip:
            195.135.221.2
        loc:
            49.4478,11.0683
        org:
            AS29298 SUSE LINUX GmbH
        postal:
            90409
        region:
            Bavaria
    changed:
        False

Calling from the SLS files

That is also supported. Naturally, you can use typical module.run if you like, but you've also got a dedicated Ansible state module. The syntax/example is the same as for new syntax for module.run. It also supports a "batch mode", or "Ansible playbook mode" where tasks can be executed from within one SLS file:

task_id:        # ID of your task
  ansible.call:  # Mandatory. This calls the modules
    - namespace.module:
      - params
    - namespace.another.module:
      - params
      - some: keywords

For example, this state will fail, because foo.bar module does not exist, as well as network.junos.junos_package requires some parameters. However it will still call everything one after another:

run_test_echo:
  ansible.call:
    - system.ping:
        - data: "Hello from Salt"
    - network.junos.junos_package:
    - foo.bar:

This results to the following output:

$ salt saltdevel state.apply ans
saltdevel:
----------
          ID: run_test_echo
    Function: ansible.call
      Result: False
     Comment: 
     Started: 15:33:04.367029
    Duration: 1409.689 ms
     Changes:   
              ----------
              foo.bar:
                  Module "foo.bar" failed. Error message: (KeyError) 'ansible.foo.bar'
              net_tools.ipinfoio_facts:
                  ----------
              network.junos.junos_package:
                  ----------
                  failed:
                      True
                  msg:
                      missing required arguments: src
              system.ping:
                  ----------
                  ping:
                      Hello from Salt

Summary for saltdevel
------------
Succeeded: 0 (changed=1)
Failed:    1
------------
Total states run:     1
Total run time:   1.410 s

Tests written?

Yes

Please review Salt's Contributing Guide for best practices.

[isbm]

@meaksh @dincamihai @cachedout — guys, Merry Christmas. 😉 Keep an eye on it (ignore lints and little bits, let it boil, I will look after a while).
@thatch45 Just like promised. 😎

[cachedout]

re-run lint

[cachedout]

This is awesome!

My main concern here is the departure from the salt_module.salt_function namespace. I think we could still maintain our current standard and just call Ansible modules with something like ansible.call ansible_module.ansible_function.

How do you feel about that, @isbm ?

[isbm]

@cachedout I feel not very well about. 😉 We actually thought about that and expected this concern, still chose to depart from that for these reasons:
— Ansible is placing everything under different directories. While it currently maintains distinct names, yet their entire structure more like objects or at least namespaced structure. That said, nobody stops any user have two modules with the same name in different places. Since Ansible module does only one thing at a time, this problem might arise pretty much soon. Moreover, these modules are from kind of "another world", where keeping them isolated is good for two another reasons: a) Users mainly will reuse their Ansible modules from Salt in order not to rewrite them. They can happily place their same-name modules in the different directories for more convenient handling etc. b) We don't need to worry for name clashes. That said, at least keeping the namespace seems to be something that is rather needed.
— The implementation option that does something like ansible.call 'some.ansible.module' looks to me pretty messy. It is better and cleaner to do Python way, e.g. in import foo.bar.blah rather then go Perl/Ruby/PHP way: require "foo.bar.blah" where that "require" would resembled "ansible.call". Another problem with this, that it gives you a feeling that the whole thing is kind of away from the Salt, while it isn't. In fact, the whole exercise is to bring the Ansible modules into the Salt, not call Ansible modules outside the Salt! And this is very important part here.
— The code is much smaller then. 😉 Basically, it introspects and installs callers into the module directly "as is", instead of keeping them away, find out, call etc.
— It is still better to have in a future:

   salt \* docker.v1.create ....
   salt \* docker.v2.create ....
   salt \* docker.v3.create ....

...rather then

   salt \* docker.create ...
   salt \* docker_ng.make ...
   salt \* docker_reallygoodthistime.construct ...
   salt \* docker_even_better.build ...

😆

On the other hand, why not depart anyway? Modules in Salt are module.function, OK. But what possibly can go wrong with module.class.function or module.submodule.function anyway as long as we have a good introspection where anyone can call "help" over "class/submodule" and see what functions are there? I find this an advantage that should not be overused, sure, but having it when it is needed isn't really a problem. Especially that Ansible caller already lists you all modules with the namespaces. You can just copypaste that and it is there.

Alas, when the entire community will still start cry "Nah, no, we do not want stick to the Salt spirit and do command target module.function options (so to say: salt \* ansible.foo.bar.spam some=options) but we love to type really long commands such as command target module.callerfunction ofwhichmodule options (so to say salt \* ansible.callmethat "foo.bar.spam" some=options) then we can always go there as well (with onion behind the cheek, crying). 😉

[gtmanfred]

Personally I do not like the departing from the module.function model. We already have other similar things that work with the whatever.call model.

Between the salt module

https://docs.saltstack.com/en/latest/ref/runners/all/salt.runners.salt.html#salt.runners.salt.cmd

And the way that most proxy minions are written.

https://github.com/saltstack/salt/blob/2017.7/salt/modules/esxi.py#L52

We already have this kind of behavior in place.

My 2¢, Thanks.

Daniel

[isbm]

@gtmanfred It is not a runner module, and the paradigm is different. When an Ansible module is called, it is called like being a part of Salt (this is a migration feature, not just generic execution implementation), while in your examples parameters are passed to somewhere else towards generic function target.

And even your example isn't the best one as it would be better to incorporate there the model I propose. Alongside of the weird path salt.runners.salt you have a) cmd and b) execute functions with the only difference that one should be remembered as for local stuff and the other one is exactly the same as previous, except works only on remote. It would be much cleaner to have "area.what.where <how>" where the what.where would replace what and anothersimilarwhat. E.g.:

salt.runners.execute.local <function> <parameters>
salt.runners.execute.remote <target> <function> <parameters>

And yes, unfortunately we have module.run instead of module.execute to the story above... 😞

[isbm]

@thatch45 thoughts?

[isbm]

@s0undt3ch do you have any ideas why tests on Ubuntu 16 fails that way? I am trying with PyTest and "traditional" tests in different directions and they are passing...

This is what happens on the Ubuntu 16:

Traceback (most recent call last):
  File "/testing/tests/unit/modules/test_ansiblegate.py", line 106, in test_resolver_module_loader_failure
    assert 'No module named ansible.modules.{0}'.format(mod) in str(import_error.value)
AssertionError

Different exception content?.. Maybe it will go away if I try now to see if the exception is raised at all. But this will kind of make the test a bit "dull" (too generic). Still I don't know how this env will react, since I am "poking the sky", in a hope it won't fail.

[isbm]

Go Go Jenkins!

[cachedout]

@isbm Right now @thatch45 is out of the office but I will get him to weigh in here as soon as possible. Thanks for your patience.

[cachedout]

What do you think about using salt.utils.timed_subprocess here and in the line below so that we can bail out if we hang?

[isbm]

Good point. Let me check for it. Getting back here ASAP.

[cachedout]

A more descriptive function name here would help make this more maintainable in the future. :]

[isbm]

OK

[isbm]

@cachedout OK, this now should be good enough.

[isbm]

@cachedout I've also added an option ansible_timeout that can be set in /etc/salt/minion defaulting to 20 minutes and a documentation. No idea why GitHub tells me this branch is out of date, I just rebased...

[isbm]

@cachedout anything else I need to do here?

[isbm]

@cachedout @thatch45 is there any chances to get this done? It is blocking further development of e.g. conversion Ansible playbooks to SLS. 😢 ...