Empty pillar variable

[mrichar1]

Hi - I'm currently trying to use gitstack following the pillarstack docs with the example of iterating over the 'roles' as shown here: https://docs.saltstack.com/en/latest/ref/pillar/all/salt.pillar.stack.html#pillarstack-configuration-files

My config looks like this:

top.sls

{% set hostname, domain = opts['id'].split('.', 1) %}

base:
 '*':
    - minions/{{ hostname }}

minions/server1.sls

roles:
  - a
  - b

stack.cfg

{%- for role in pillar.get('roles', []) %}
roles/{{ role }}.sls
{%- endfor %}

When I call salt 'server1' pillar.items I see:

server1:
    ----------
    roles:
        - a
        - b

However none of the data in roles/a.sls or roles/b.sls is ever included.

I wasn't sure the best way to debug the stack state, so for testing I put the following in stack.cfg:

TESTING/{{ pillar }}

This results in the following in the logs:

Ignoring pillar stack template "TESTING/{}" ...

Any ideas why pillar is an empty dict?

[amendlik]

Would it be possible for you to test your configuration under PillarStack?
GitStack is just a wrapper around PillarStack, so we need to make sure that is working first.

[mrichar1]

I've updated the master config to have:

ext_pillar:
  - stack: /etc/salt/stack/stack.cfg

And dropped the above files into /srv/pillar and /etc/salt/stack/ as appropriate.

Now I see {{ pillar }} being expanded properly in the logs:

Ignoring pillar stack template "TESTING/{u'roles': [u'a', u'b']}" ...

And the for seems to be looping as expected - if I create roles/a.sls then it's pillar contents ends up being processed correctly and appear when calling pillar.items.

So it does look like a gitstack issue I'm afraid!

[mrichar1]

OK - I just managed to solve this one.

It turns out that if you have stack config and pillars in the same git repo, then you must put the git entry before the gitstack entry in the master ext_pillar section:

ext_pillar:
  - git: 
    - master http://example.com/salt.git
  - gitstack:
      - master http://example.com/salt.git:
      - stack: _stack/stack.cfg

I'm guessing what happens is that ext_pillar parts are processed in order, and that otherwise gitstackruns 'before' top.sls has been read in from the git repo, so gets an empty pillar. However top.sls is then read in subsequently, which is why the roles defined in it do end up in the pillar by the time we call pillar.items. Having swapped them round it all works as expected.

I'm guessing this will never have been spotted if top.sls is local, or in a different git repo included before gitstack?

Could you update the documentation to reflect this? (There's also a minor bug in that the configuration yaml for gitstack in ext_pillar is missing a colon).

[amendlik]

That makes sense. It wasn't clear to me that you were using the Git pillar. I assumed you were using the standard, built-in, pillar and I was having a very hard time understanding the issue.

[amendlik]

@mrichar1 I've added some documentation on pillar chaining. Please let me know if this looks sufficient.
https://github.com/amendlik/gitstack-pillar/blob/develop/docs/configuration.md

[mrichar1]

That looks great - thanks!