saltmod.state state module function does not take in custom pillar

[felix-deschamps]

Unlike the state.sls execution module function, the saltmod.state function (salt\states\saltmod.py) function does not take in a custom pillar. This precludes scenarios where custom pillar can be passed in an orchestration. So for example, you can invoke an orchestration the following way:

salt-run state.orchestrate my_orchestration pillar='{p1: value1,p2: value2}'

and since the state.orchestrate function takes in a custom pillar you could then do:

{% set p1 = salt['pillar.get']('p1','default1') %}
{% set p2 = salt['pillar.get']('p2','default2') %}
{% set target = salt['pillar.get']('target') %}

my-orchestration:
    salt.function:
        - name: state.sls
        - tgt: '{{ target }}'
        - arg:
          - state1
        - kwarg:
            pillar:
              p1: '{{ p1 }}'
              p2: '{{ p1 }}'

But you would not be able to do the same with the 'salt.state' function, since it doesn't support a custom pillar. We should enable it so you could invoke custom pillar from either one.

This is by the way very useful if you are doing peer communication and you want to invoke a runner from a client but want to pass in custom parameters.

[basepi]

So the fact that the salt.function state allows for custom pillar in this instance is not because of anything that salt.function does. Rather, the state.sls execution function is what takes the custom pillar in the pillar argument. Similarly, the salt.state state allows you to pass a pillar argument into any state which allows for such an argument. So what you're asking for is not enabling custom pillar in the salt.state state function, but rather in the state function that it targets.

Can you give a particular example of how you would want to use this so we can discuss adding support for your use case?

[khrisrichardson]

This is a duplicate of #11904.

In my particular case, I wanted to run an orchestrate runner that called a generic state, fittingly called "orchestrate", which determines what "related" states to run as a function of highstate and custom pillar data.

My reasoning is to keep the highstate lean and relevant only to the role(s) running on the node, whilst only running "related" states as the need arises in an orchestrated or event-driven manner via the reactor. This is in order to have something akin to Juju's relation hooks. I have ultimately settled for just using the reactor, although I regularly get jobs stuck in the queue, and am not opposed to the idea of periodically triggering an orchestrate runner.

As an example of how I wished to use the orchestrate runner, if I started a node with a logstash role, aside from setting up logstash, I also wanted to broadcast to the remaining nodes that there's a new logstash node to which they can now point their related services....

state_sls_orchestrate_logstash:
  salt.state:
    - tgt:        'G:environment:{{ environment }} and not G@roles:logstash'
    - tgt_type:    compound
    - sls:         orchestrate
{# missing desired functionality
    - kwarg:
        pillar:
          related: {'roles': ['logstash']}
#}

I have structured my state files so that related states reside in salt/*/relate-*.sls. For the salt-master to relate to logstash by way of editing /etc/salt/master.d/log.conf, for instance, I could either run...

salt-call state.sls salt-master.relate-logstash

or...

salt-call state.sls orchestrate pillar="related: {'roles': ['logstash']}"

With the former I am required to know in advance which states are present that have a relationship to logstash, while with the latter using my kludgily written orchestrate state file, which I am posting despite fears of ridicule for not yet dropping the logic into an execution module (it's in the works, I swear), I can determine which states are in the highstate, and which of them have an equivalent relate-${role}.sls file...

{% set includes       =  [] %}
{% set nodes          =  [] %}
{% set roles          =  [] %}

{% set related        =  salt['pillar.get']('related', {}) %}
{% if  related.roles is  sequence %}
{% set roles          =  related.roles %}
{% else %}
{% if  related.node  is  defined %}
{% set node           =  related.node  %}
{% set environment    =  salt['mine.get'](node, 'grains.item')[node]['environment'] %}
{% if  environment   ==  salt['grains.get']('environment') %}
{% do  nodes.append(node) %}
{% endif %}
{% else %}
{% set environment    =  salt['grains.get']('environment') %}
{% for node          in  salt['mine.get']('environment:' + environment, 'grains.item', expr_form='grain').keys() %}
{% do  nodes.append(node) %}
{% endfor %}
{% endif %}
{% for node          in  nodes %}
{% for role          in  salt['mine.get'](node, 'grains.item')[node]['roles']|sort %}
{% if  role not      in  roles %}
{% do  roles.append(role) %}
{% endif %}
{% endfor %}
{% endfor %}
{% endif %}

{% set states         =  salt['cp.list_states']() %}
{# hurray, infinite loop! #}
{# set highstate      =  salt['state.show_highstate'](pillar={'orchestrate': True}, queue=True) #}
{% set highstate      =  salt['state.show_highstate'](queue=True) %}
{% if  highstate     is  mapping %}
{% for high, data    in  highstate|dictsort %}
{% if  data          is  mapping %}
{% set sls            =  data.get('__sls__') %}
{% for role          in  roles|sort %}
{% set state          =  sls + '.relate-' + role %}
{% if  state not     in  includes
   and state         in  states %}
{% do  includes.append(state) %}
{% endif %}
{% endfor %}
{% endif %}
{% endfor %}
{% endif %}

{% if includes %}
include:
{% for include in includes|sort %}
  - {{ include }}
{% endfor %}
{% else %}
::
  cmd.run:         []
{% endif %}

[basepi]

I just realized that I was making assumptions about salt.state and you're right, you can't pass in custom pillar even if the states can use it.

That said, I'm going to close this in favor of #11904.

[felix-deschamps]

Works for me. The way khrisrichardson is planning to call salt.state is pretty much how i expect to call it as well (similar scenario for me). Happy to see it fixed the way he suggests. Though in my case i'm using salt.state from an orchestrate job, but functionally it should be the same thing ...