Failed to load ext_pillar stack: no filter named 'unique' in 2018.3

[morgana2313]

Description of Issue/Question

After upgrading to 2018.7 Saltstack doesn't know the Jinja filter 'unique' in the ext_pillar stack.
This worked ok before upgrading from 2017.7 and after downgrading back to 2017.7.

local:
    Data failed to compile:
----------
    Pillar failed to render with the following messages:
----------
    Failed to load ext_pillar stack: no filter named 'unique'

Setup

snippet from external pillar stack *.yml file:

{% for app in a_dict | unique %}
{% endfor %}

[Ch3LL]

can you please share how you have your external pillar setup?

[morgana2313]

the master.conf contains:

ext_pillar:
  - stack:
      - [somepath]/stack.cfg
      - [somepath]/stack2.cfg

the stack2.cfg file contains:

somefile.yml

the somefile.yml contains:

        {% for app in stack.something | unique %}  # This 'unique' jinja filter gives an error in saltstack 2018.03??
          {{ app }}: {{ app }}
        {% endfor %}

[Ch3LL]

Thanks for adding teh configuration. Helped me to replicate it.

bisect shows: 494022d which is this PR.

ping @lorengordon any ideas here?

[lorengordon]

Basically the same problem fixed by #46744. The utils modules were all shifted around... The modules containing the jinja filters are not getting imported, and so the decorator is not getting called, which means the filters in utils modules that are not imported are not getting injected into the jinja env.

[morgana2313]

This problem still exists in Saltstack 2018.3.2

[Oloremo]

Having a similar problem with pillarstack and to_bool Jinja filter at 2018.3.3. Any updates?

[Ch3LL]

its not currently assigned out. If anyone wants to try to push a PR we would definitely appreciate it.

[Oloremo]

@Ch3LL Any updates? :-(

[Ch3LL]

not any updates currently. does anyone from @saltstack/team-core have time to take a look at this one?

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

[Oloremo]

still a very irritating issue.

[stale]

Thank you for updating this issue. It is no longer marked as stale.

[gbunt]

Does this affect any ext pillar? In any case, when using pillarstack it seems none of the filters are available, is there are a target release for this yet? This is 2019.2.0 anyway;

{% set string = 'test' %}
test:
- {{ string | md5 }}
- {{ string | base64_encode }}
- {{ ['foo', 'foo', 'bar'] | unique }}

Results:

Failed to load ext_pillar stack: no filter named 'md5'
Failed to load ext_pillar stack: no filter named 'base64_encode'
Failed to load ext_pillar stack: no filter named 'unique'

Seems to me this affects a whole lot of installs? Have to revert to cmd.shell now. Not too pretty :-/

[sagetherage]

@Oloremo I don't know that this can get done for the Sodium release, but I will put it in the queue to see if we can get it assigned. Very likely it will need help from the community that does know this part of the code base, but I will try to find someone.

[Oloremo]

@sagetherage I have a hint that it could be pillarstack related and I try to validate this and maybe patch in pillarstack upstream

[daks]

It seems that this problem still exists in pillarstack in Salt 3004.2

[derdeagle]

I think i just stumbled upon this issue in 3005.1.

Failed to load ext_pillar gitstack: No filter named 'regex_replace'.

The line in question is as follows where host is something like test01.example.com and I want to only have test.

{% set part = host|regex_replace("([a-zA-Z0-9-]+)[0-9]{2}\.", "\\1") %}