allow pillar files to 'include'/load other pillar files.

[inthecloud247]

A few times in the last few months I've wanted to have pillar values that I could 'include' or load into my pillar files. The problem is that the pillar dict doesn't seem to be available at all until the pillar has completely finished loading, and pillar files don't seem to be able to include others.

One example: I have a live_instances.sls yaml file that contains a list of all my servers. I need to load this live_instances.sls file and only add to the pillar the servers that match a regex on their fqdn.

live_instances.sls:

aws:
  zebra-staging.test.net:
    id: i-3e95670c
    image: null
    private_ips:
    - 10.252.178.30
    public_ips:
    - 54.243.237.177
    size: null
    state: RUNNING
  yak-staging.test.net:
    id: i-2382ba10
    image: null
    private_ips:
    - 10.248.46.125
    public_ips:
    - 54.245.99.206
    size: null
    state: STOPPED

server_status.sls:

include:
  - live_instances.sls

servers:
% for server in pillar['aws']:
  % if pillar['aws']['server']['state'] == 'RUNNING':
  ${server}:
    state: ${pillar['aws']['server']['state']}
  % endif  
% endfor

I guess this can be accomplished currently by natively including the pillar files by using your templating language such as mako or jinja, but that seems like a hack and salt should be able to handle this natively.

I thought that maybe instad of allowing 'includes', it might be a better idea to create a 'high pillar' or some kind of pillar that is loaded before the main pillar begins to load so that the dict is available. Either way would work.

[inthecloud247]

related to #4326

and discussed here: https://groups.google.com/d/topic/salt-users/gH7DHC0Ck88/discussion

[erchn]

+1

[drawsmcgraw]

+1

[thecodeassassin]

It is stated that this should work in the docs. But it doesn't work.

[eliasp]

@thecodeassassin Including pillars using include doesn't mean, their values are available to be re-used in the pillar including them, but that instead you don't have to assign the included pillar to a target.

So instead of:

base:
    'node-*-web':
        - pillars.db_credentials
        - pillars.admin_list
        - pillars.web_app

You could simply include the db_credentials and admin_list in web_app and then only assign web_app to node-*-web without having to worry about the other pillars.

Re-using values of other pillars inside a pillar file is not yet supported.

[johtso]

This feels like quite a big issue when you start working with formulas.

Because formulas use pillar data to describe everything, you then have no way to define and use values in a DRY way on top of that.

For example if you're using the users formula, and you have two users with the same ssh public key, there's no way to define that key in one place and reference it in two places in the pillar data.

[alexhayes]

+1

[aviau]

no way to define that key in one place and reference it in two places in the pillar data.

Indeed! This is pretty basic :( I don't really understand how people use Salt if that is not possible....

[zerthimon]

+1

[titilambert]

+1 @aviau ;)

[ghost]

+1

[h0jeZvgoxFepBQ2C]

+1

[AndreiPashkin]

It's impossible to implement it that way, because to handle inclusion, you need to render a pillar file first, using a renderer (e.g. Jinja2).

[bobrik]

Pillars can include other pillars:

{%- import_yaml "labels/init.sls" as labels %}

Here labels/init.sls is a pillar with jinja2 processing. However, you should be careful, because import_yaml does processing for the included file from scratch each time you use it. For small enough pillars it's possible to cache pillar values into grains object and reuse that.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.