Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Wildcard inside top.sls file for pillar #22063

Closed
jeanpralo opened this issue Mar 26, 2015 · 37 comments

Comments

@jeanpralo
Copy link
Contributor

commented Mar 26, 2015

NOTE: Works for states, this is now an issue for adding this functionality for pillar

Hiy guys,

I am trying to use a wildcard feature documented here :

http://docs.saltstack.com/en/latest/topics/releases/0.9.9.html 7.2.49.1.9. Wildcards for SLS Modules

So I wrote this simple top.sls states inside /srv/pillar:

base:
  'ssh*':
    - users.*

The directory /srv/pillar/user directory contains two files user1.sls and user2.sls

But when trying to get the pillar items on the salt master I am getting an error :

ssh-1:
    ----------
    _errors:
        - Specified SLS 'users.*' in environment '58faaa5-dev' is not available on the salt master

Here is the version of salt I use :

root@salt-master:~/work# salt --version
salt 2015.2.0-742-g58faaa5 (Lithium)

I have asked the question on irc, the guys seemed to think that feature might have been killed.

Thanks for your help.

Cheers,

@basepi

This comment has been minimized.

Copy link
Collaborator

commented Mar 27, 2015

Have you attempted to use this with the state system instead of the pillar system? From the release notes, it appears this was implemented for the state system, but it may not have been implemented for the pillar system.

@basepi basepi added the Info Needed label Mar 27, 2015
@basepi basepi added this to the Blocked milestone Mar 27, 2015
@jeanpralo

This comment has been minimized.

Copy link
Contributor Author

commented Mar 29, 2015

Hiya,

Tested with the states works fine. Thought that the state system and pillar were at the same level of functionality, my mistake.

Would be nice to have it for the pillar system too.

Thanks anyway.

@iggy

This comment has been minimized.

Copy link
Contributor

commented Mar 30, 2015

Would also be nice to document the differences between state and pillar top files.

@basepi

This comment has been minimized.

Copy link
Collaborator

commented Mar 31, 2015

Both great points. I've marked this as a feature.

@basepi basepi added Feature and removed Info Needed labels Mar 31, 2015
@basepi basepi modified the milestones: Approved, Blocked Mar 31, 2015
@basepi basepi changed the title Wildcard inside top.sls file Wildcard inside top.sls file for pillar Mar 31, 2015
@aboe76

This comment has been minimized.

Copy link
Contributor

commented Jun 8, 2015

@basepi is this still on the wish list? I hope it isn't forgotten.... :)

@basepi

This comment has been minimized.

Copy link
Collaborator

commented Jun 8, 2015

Not forgotten, just busy. =)

@tehmaspc

This comment has been minimized.

Copy link

commented Sep 25, 2015

+1 - just came across this because i thought i could do the same :)

@pduersteler

This comment has been minimized.

Copy link

commented Nov 24, 2015

👍 also from our side

@michaelkarrer81

This comment has been minimized.

Copy link

commented Dec 16, 2015

+1 - really needed - Any possible workaround for now?

@wych42

This comment has been minimized.

Copy link
Contributor

commented Dec 17, 2015

+1. need this feature.

@andre-brongniart

This comment has been minimized.

Copy link

commented Jan 25, 2016

+1 would be an awesome feature. Not sure what workaround to use other than what is recommended to "use a little Jinja and the file.readdir module you can loop over the contents of a directory and dynamically include them."... from https://groups.google.com/forum/#!topic/salt-users/QoYkSwyENiI
Not sure what they meant...

@sjorge

This comment has been minimized.

Copy link
Contributor

commented Feb 21, 2016

i was under the impression it was not working in pillar and states. I will need to retest this later.
It would indeed be nice to have!

Edit: works fine for states in 2016.3/develop, just checked.

@mallniya

This comment has been minimized.

Copy link

commented Apr 14, 2016

👍 just stumbled upon the need of such a feature.

@shellhacks

This comment has been minimized.

Copy link

commented Jul 22, 2016

+1

4 similar comments
@jwyoungpm

This comment has been minimized.

Copy link

commented Aug 23, 2016

+1

@onlyanegg

This comment has been minimized.

Copy link
Contributor

commented Sep 2, 2016

+1

@mooperd

This comment has been minimized.

Copy link

commented Sep 22, 2016

+1

@danielmorlock

This comment has been minimized.

Copy link

commented Oct 21, 2016

+1

@pierluca

This comment has been minimized.

Copy link

commented Nov 16, 2016

+1

@jwyoungpm

This comment has been minimized.

Copy link

commented Dec 14, 2016

If anyone is interested, I'm using the following scripts to manage this at the deploy time of some of our applications:

https://gist.github.com/jwyoungpm/f74091ac41733deddf7f348f83a020fc

Keep in mind, this is relatively rudimentary and assumes that you have a folder called 'app' under /srv/pillar. This also assumes you have that app folder included in your top.sls:

/srv/pillar
   ->top.sls (has an include for the app folder)
   ->app
      ->init.sls

@nicomz

This comment has been minimized.

Copy link

commented Dec 19, 2016

+1. Would be very useful

@porunov

This comment has been minimized.

Copy link
Contributor

commented Jan 3, 2017

+1

@pepoluan

This comment has been minimized.

Copy link

commented Jan 24, 2017

My workaround is by creating a custom module cat.py as follows:

#!/usr/bin/env python
# Module name: cat.py
from __future__ import print_function, unicode_literals

import os
import fnmatch


# noinspection PyShadowingBuiltins
def file(name, indent=0):
    ind, ret = ' ' * indent, []
    with open(name, 'rt') as fin:
        for ln in fin:
            ret.append('{0}{1}'.format(ind, ln.rstrip('\r\n')))
    return ret


def directory(name, recurse=False, indent=0, file_filter='*', dir_filter='*'):
    ret = []
    for root, dirs, files in os.walk(name):
        root_base = os.path.basename(root)
        if fnmatch.fnmatch(root_base, dir_filter):
            for fn in files:
                if fnmatch.fnmatch(fn, file_filter):
                    ret.extend(file(os.path.join(root, fn), indent=indent))
        if not recurse:
            break
    return ret

Then, in the proper location in the .sls file, add the following jinja:

{% for ln in salt.cat.directory( ... ) %}
{{ ln }}
{%- endfor %}

EDIT: The latest version will always be available on my Github repo: https://github.com/pepoluan/saltstack-custom-modules

@kapucko

This comment has been minimized.

Copy link

commented Feb 21, 2017

+1

@gerardjp

This comment has been minimized.

Copy link

commented Mar 17, 2017

+1

@wych42

This comment has been minimized.

Copy link
Contributor

commented May 11, 2017

Seems this feature is already supported: https://docs.saltstack.com/en/latest/topics/pillar/

@lorengordon

This comment has been minimized.

Copy link
Contributor

commented May 11, 2017

@marlowew I'm only seeing info there on globbing the targets, not the pillar paths.

Works (but not what this issue is about):

base:
  'foo*':
    - bar

Not works (what this issue is about):

base:
  'foo':
    - bar*

Did I miss something in that link?

@wych42

This comment has been minimized.

Copy link
Contributor

commented May 12, 2017

@lorengordon my mistake, I will have a try to solve this issue.

@wych42

This comment has been minimized.

Copy link
Contributor

commented Jun 12, 2017

fixed in #41423

@aboe76

This comment has been minimized.

Copy link
Contributor

commented Jun 12, 2017

@marlowew nice thanks for this!

@PhilippeAB

This comment has been minimized.

Copy link

commented Oct 20, 2017

The fix in #41423 don't work for me

As a workaround you can create a folder with all your wildcard wanted file and create and init file with include for all the file in the folder preceded with the full path to the file
example
folder ./users/
with user1.sls, user2.sls
create init.sls file in ./users/ with
include:

  • users.user1
  • users.user2
@anitakrueger

This comment has been minimized.

Copy link
Contributor

commented Dec 8, 2017

Is this really fixed with #41423?
I can't make it work either, with exactly the same use case (users) but using an include:

    _errors:
        - Specified SLS 'common.users.present.*' in environment 'base' is not available on the salt master

This is my include statement:

include:
  - common.users.present.*
@anitakrueger

This comment has been minimized.

Copy link
Contributor

commented Dec 8, 2017

@marlowew I've tested this again and it looks like this works from the top file, but not in includes.
So if my top.sls looks like this, the * gets matched correctly:

base:
  '*':
    - common.users.present.*

If my top file just has the pillar file that then has the include, it doesn't match the *.

What would it take to fix this for includes do you think? Why are they different than matches in the top file?

@wych42

This comment has been minimized.

Copy link
Contributor

commented Dec 9, 2017

@anitakrueger I haven't use this feature for months, but I will look into this next week.

@amontalban

This comment has been minimized.

Copy link
Contributor

commented Dec 29, 2017

I also want to use it in include statements. Did you have time to look in to this @marlowew?

Thanks!

@wych42

This comment has been minimized.

Copy link
Contributor

commented Jan 4, 2018

@amontalban Just pushed a PR #45269 , Still working on passing CI, but I have test it with a simple test server and worked.

@amontalban

This comment has been minimized.

Copy link
Contributor

commented Jan 8, 2018

Thanks @marlowew, will give it a try!

@rallytime rallytime closed this in fbab73a Jan 26, 2018
rallytime added a commit that referenced this issue Jan 26, 2018
Fix #22063: pillar wildcard support include
dwoz added a commit to dwoz/salt that referenced this issue Apr 10, 2019
This reverts commit fbab73a.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
You can’t perform that action at this time.